Building a coding frame that holds, when counting is legitimate, and how content analysis differs from thematic analysis.
Content analysis is the systematic categorisation of text, media or documents using a coding frame that is defined in advance and applied consistently. Its strength is replicability: another researcher with your frame should reach broadly the same result. It comes in two traditions — a quantitative one that counts categories and reports agreement between coders, and a qualitative one that stays interpretive — and saying which you are doing is the first thing a methods chapter has to establish.
Much of the confusion around content analysis comes from the term covering two related but different practices.
Quantitative content analysis treats coding as measurement. Categories are defined in advance in a codebook, applied by more than one coder, and the agreement between them is calculated and reported. Results are usually frequencies, sometimes analysed statistically. This tradition is well established in media and communication research, and Krippendorff’s work is the standard methodological reference.
Qualitative content analysis keeps the systematic coding frame but treats the output as interpretive description rather than measurement. Hsieh and Shannon’s widely cited paper distinguishes three forms: conventional, where categories are derived from the data; directed, where an existing theory supplies the starting categories and the analysis extends or challenges it; and summative, which begins by counting particular words or content and then interprets the underlying context.
These differ in what they require of you. The quantitative tradition expects reliability statistics; the qualitative tradition generally does not, and reporting one there can look like machinery borrowed without reason. Name the tradition and the specific source you are following.
It fits when your question is about what is present in a body of material and how consistently, and when someone else should be able to repeat the categorisation and get the same answer.
It suits:
It fits badly when the question is about meaning that has to be interpreted in context, when the material is a small number of rich interviews, or when the interesting thing is what is not said. Frames capture presence well and absence poorly.
The coding frame is the instrument, and the analysis is only as good as it is. Most content analyses that fail do so here rather than in the coding itself.
For every category: a name, a definition, an indication of when it applies, an indication of when it does not, and at least one example from the material. The “when it does not” line is the one people leave out, and it is where most disagreement between coders originates.
Two requirements are conventionally placed on a frame. Categories should be mutually exclusive within a dimension — a unit should not sit equally well in two — and the set should be exhaustive, covering everything you will encounter, which usually means an explicit “other” category that you then monitor. If “other” fills up, the frame is wrong and needs revising.
Specify what is being coded: the word, the sentence, the paragraph, the whole article, the speaking turn. Different units give different results from identical material, and a chapter that does not state its unit cannot be replicated. Where the unit is meaning-based rather than structural — “each distinct argument” — the rule for identifying boundaries has to be stated too.
Apply the draft frame to a subset of material, ideally with a second coder, and revise. Expect several rounds. This is normal and should be described in the chapter rather than hidden — a frame revised through piloting is a strength, provided all material is then coded under the final version rather than early items being left coded under an earlier draft.
Where coding is treated as measurement, showing that the frame can be applied consistently by more than one person is what makes the results credible. Simple percentage agreement overstates consistency, because some agreement happens by chance; the standard measures correct for that.
Cohen’s kappa handles two coders with categorical data. Krippendorff’s alpha is more flexible — it copes with more than two coders, missing data and different levels of measurement — which is why methodological texts on content analysis tend to favour it.
What counts as an acceptable value is genuinely contested, and published guidance varies between fields and between sources. So report the coefficient, say which one it is, say how much material was double-coded and how it was selected, and justify your threshold with a citation rather than presenting a number as though it were a universal standard. If agreement is low, the honest response is to revise the frame and recode — not to report the figure and move on.
One caution worth keeping in view: high agreement shows the frame is applied consistently. It does not show the frame captures anything worth capturing. A trivial frame can be coded almost perfectly.
Content analysis is the qualitative-adjacent method where counting is defensible, because the design is built for it: a systematic frame applied to a defined corpus supports statements about how often something appears in that corpus.
Two conditions have to hold. The corpus must be defined by a stated sampling rule — all policy documents of a given type in a given period, say — rather than assembled opportunistically. And the claim must stay inside the corpus: how often something appears in your sample of newspapers is a statement about those newspapers, not about public opinion.
Frequency also needs interpreting rather than reporting. That a term appears often may mean it is important, or that it is a required formulation nobody thinks about. Summative content analysis exists precisely because counts need contextual interpretation to mean anything.
A method for systematically categorising text, media or documents using a coding frame defined in advance and applied consistently across the material. Its defining strength is replicability — another researcher applying your frame should reach broadly the same result — which is why the frame, the coding unit and the corpus have to be specified precisely.
Content analysis is systematic and replicable, works from a coding frame usually set before the main coding, and often reports how often categories occur. Thematic analysis is interpretive, develops themes through the analysis itself, and generally avoids counting. Choose content analysis when presence and prevalence in a defined body of material are part of the finding; choose thematic analysis when the question is what something means.
In the quantitative tradition, yes — coding is treated as measurement, so consistency between coders is what makes results credible. In qualitative content analysis it is generally not expected, and reporting it can look like borrowed machinery. If you do report it, use a chance-corrected measure such as Cohen’s kappa or Krippendorff’s alpha, state how much material was double-coded and how it was selected, and cite your source for the threshold you treat as acceptable, since guidance on that genuinely varies.
In content analysis, yes, because the design supports it: a systematic frame applied to a defined corpus licenses statements about how often something appears in that corpus. The claim has to stay inside the corpus, and the count still needs interpreting — frequent use of a term may signal importance, or merely a required form of words. In interpretive approaches such as reflexive thematic analysis, counting is generally inappropriate.
Choose the smallest unit that can carry the meaning you are coding for, and state it explicitly. Word-level units suit vocabulary questions; sentences or paragraphs suit arguments and claims; whole documents suit questions about overall framing. Where the unit is defined by meaning rather than structure, state the rule for identifying its boundaries. Different units produce different results from the same material, so this cannot be left implicit.
A frame with overlapping categories or an unstated unit cannot be repaired after the corpus has been coded. Describe your material and your draft frame, and a PhD in your field will stress-test it.
Leave an email or a WhatsApp number — whichever you prefer — and tick how we should reach you. We reach out within 30 minutes.
