科研 Skill
内容分析编码方法
content-analysis-methodology
Designs codebooks, sampling plans, and intercoder-reliability checks for systematic content analysis. Use when measuring patterns in media coverage or validating an LLM classifier.
SKILL.md
Content analysis methodology
A systematic research method for measuring patterns in media content, text, images, or video at scale, the core method of academic communication research and a useful complement to data-journalism's data-oriented methods when the "data" is unstructured text or media rather than a spreadsheet.
Purpose
Content analysis answers questions like "how often did cable news mention inflation in election years" or "do local TV crime stories disproportionately feature Black suspects relative to arrest-rate data" with a reproducible, quantifiable method rather than a reporter's impression. The method's credibility rests entirely on two things this skill is built around: a codebook precise enough that two independent coders agree most of the time, and a sampling plan that supports the claim being made.
When to use
- Measuring how often, how prominently, or in what framing a topic, person, or group appears across a body of media content (articles, broadcasts, social posts, press releases).
- Testing a specific hypothesis about bias, framing, or representation in coverage (e.g., source diversity, tone, topic emphasis) that needs to stand up to methodological scrutiny.
- Academic or think-tank communication research requiring a citable, replicable method section.
- Validating an AI/LLM-based classification pipeline against a human-coded gold standard before trusting it at scale.
When not to use
- A single reporter's close reading of a handful of articles for a narrative story, that's ordinary reporting, not content analysis, and dressing it up with this method's vocabulary overstates its rigor.
- When the underlying claim is really about structured data (contract awards, campaign spending) rather than about patterns in unstructured content, use data-journalism instead.
Step 1: Define the unit of analysis
Decide precisely what gets coded as one unit before sampling anything:
| Unit type | Example | Use when |
|---|---|---|
| Whole story/article | One newspaper article | Measuring topic presence, overall tone |
| Paragraph | One paragraph within an article | Fine-grained framing analysis |
| Sentence/clause | One sentence | Claim-level or quote-level coding |
| Source/quote | Each quoted or named source | Source diversity, who gets to speak |
| Image/frame | One photo, one video frame or shot | Visual framing, representation analysis |
| Segment | One broadcast segment or social post | Cross-platform comparison |
Mixing units mid-study (coding some things by article, others by paragraph) breaks comparability. Pick one level and code everything at that level.
Step 2: Define the population and draw the sample
- Population: the full set of content the claim is about (e.g., "all evening broadcasts on the three major US networks during the 2026 election cycle"). State it precisely enough that someone else could reconstruct it.
- Sampling frame: the actual retrievable source (a news database like Nexis Uni, GDELT, a network's own archive, a platform API) — note where the frame diverges from the true population (e.g., a paywalled archive that excludes some outlets, a platform API that doesn't return deleted posts).
- Sampling method: census (code everything, feasible only for small populations), simple random sample, or constructed-week sampling (a classic technique for news content: sample one Monday, one Tuesday, etc. from different weeks to control for day-of-week publishing patterns, more efficient than pure random sampling for periodicity-heavy content).
- Sample size: there's no universal rule, but report the margin of error or confidence interval for any population-level claim based on a sample, and state the final N actually coded (after excluding duplicates, irrelevant hits, and paywalled/inaccessible items).
Step 3: Build the codebook
The codebook is the single most important artifact, it is what makes the study replicable. For each variable:
## Variable: [name]
**Definition:** [precise, operational definition — not a vague concept]
**Values:** [exhaustive, mutually exclusive category list, or numeric scale]
**Decision rules:** [how to handle ambiguous cases, with examples]
**Examples:**
- [example unit] → coded as [value], because [reasoning]
- [borderline example] → coded as [value], because [reasoning, explicitly
addressing why it's not the other plausible value]
Good codebooks spend most of their length on decision rules for borderline cases, not on the easy cases. If coders keep disagreeing on a particular type of case during pilot coding, that's a signal the codebook needs a new rule, not that the coders need more training.
Pilot the codebook on a small sample (20–50 units) with at least two coders, revise based on where they disagree, then freeze it before the real coding begins. Changing the codebook mid-study invalidates earlier coding.
Step 4: Establish intercoder reliability
Before trusting single-coder results (or an LLM classifier) at scale, measure agreement between two or more independent coders (or a coder and a validated gold standard) on a shared subsample, typically 10–20% of the full sample.
Why simple percent agreement is misleading: if 90% of content falls into one category, two coders who both just guess that category will "agree" 81% of the time by chance alone. Use a chance-corrected statistic instead:
| Statistic | Handles | Common threshold |
|---|---|---|
| Cohen's kappa | Two coders, nominal categories | κ ≥ 0.61 "substantial", ≥ 0.80 "excellent" (Landis & Koch 1977 scale, widely cited but treat as a rough guide, not a law) |
| Krippendorff's alpha | Any number of coders, nominal/ordinal/interval/ratio data, missing data | α ≥ 0.80 for confident conclusions, 0.667 the lowest defensible floor for tentative conclusions (Krippendorff's own thresholds, widely adopted in communication research) |
| Scott's pi | Two coders, nominal, assumes same marginal distribution | Rarely preferred over kappa/alpha today |
Tools:
- R:
irrpackage (kappa2(),kripp.alpha()), orDescTools. - Python:
nltk.metrics.agreement.AnnotationTask(for kappa/alpha),krippendorffpackage (pip install krippendorff) for a dedicated, well-tested Krippendorff's alpha implementation that handles missing data. - Report per-variable reliability, not just one overall number, a codebook with ten variables can have excellent reliability on easy variables (does the story mention a named topic) and poor reliability on hard ones (overall tone), and readers need to know which is which.
Step 5: Code the full sample and analyze
- Once reliability is established on the pilot subsample, code the remainder (single-coder is standard practice at this stage if reliability was validated; double-code a rotating subsample throughout the full study to catch coder drift over a long project).
- Analyze with the unit of analysis as the unit of statistical analysis, not the raw sentence count, if the unit is "story," run statistics on per-story codes, don't pool to the sentence level without re-justifying the unit.
- Report frequencies, cross-tabulations, and, where a comparison is being made (e.g., outlet A vs. outlet B), the appropriate test (chi-square for categorical comparisons, t-test/ANOVA for means) with effect sizes, not just p-values.
Using LLMs for content analysis (and validating them)
LLMs can code large corpora faster than human coders, but every automated classification pipeline needs the same reliability validation as a human coder:
- Hand-code a gold-standard subsample (100+ units) using the frozen codebook.
- Run the LLM classifier on the same subsample with the codebook definitions in the prompt.
- Compute agreement (kappa or alpha) between the LLM output and the human gold standard, using the same thresholds as Step 4.
- Only proceed to full-corpus LLM coding if agreement clears the same bar you'd require of a human coder; report the validation statistic in any published methodology, not just "we used an LLM to code this."
- Re-validate on a fresh subsample if the prompt, model, or corpus changes materially, a validated pipeline is validated for that configuration, not forever.
Quality checks
- Unit of analysis defined and held constant throughout.
- Population and sampling frame explicitly stated, including known gaps.
- Codebook frozen before full coding began, with a version/date on it.
- Intercoder reliability reported per variable, using a chance-corrected statistic (kappa or alpha), on an independently coded subsample.
- If an LLM did the coding, its agreement with a human gold standard is reported using the same statistic and threshold as a human coder would need to clear.
- Final N reported after exclusions, with reasons for exclusion stated.
- Any comparative claim (outlet A covers X more than outlet B) backed by a statistical test, not just a raw count difference.
Prerequisites
- Requires the content corpus to be legally and practically accessible (a news database subscription, platform API access, or an internal archive); budget time for retrieval before coding can start.
- Requires at least one other person to serve as a second coder for the reliability check, a single-coder study with no reliability check is a much weaker form of evidence and should be labeled as such.
- Basic statistics (chi-square, correlation, chance-corrected agreement) is needed to interpret and report results correctly.
Limitations
- Content analysis measures manifest content (what's observably there), not reception or effects, a finding that a topic appears more often in one outlet doesn't by itself establish that it changed anyone's opinion. Don't overclaim effects from a pure content measure.
- Reliability statistics measure whether coders agree with each other (or an LLM agrees with coders), not whether the codebook itself is theoretically valid, a high kappa with a poorly conceived codebook still produces a rigorous measurement of the wrong thing.
- Constructed-week and other periodicity-aware sampling methods are tuned for traditional news-cycle content; they transfer imperfectly to platforms with different temporal dynamics (a viral social post's lifecycle doesn't follow a weekly news cadence), adapt the sampling design to the medium.
Related skills
- data-journalism, Statistical analysis once content has been coded into structured data
- social-media-intelligence, Collecting the social-media corpus this method would then code
- academic-writing, Writing up methods and results for publication
示例提示词
帮我设计一套编码本并计算编码员间信度来分析这批报道样本。
所属插件
安装该插件会一并获得同领域的 19 个 Skill / MCP。
同插件的其他科研 Skill
- 学术写作规范合规Guides scholarly writing, research design, citation, grant proposals, and AI-disclosure compliance. Use when drafting papers, preregistering studies, or checking CRediT and open-access rules.
- 危机公关快速响应Provides rapid-response frameworks, escalation levels, and templates for breaking news and misinformation. Use when coordinating fast fact-checking or crisis statements on deadline.
- 数据新闻分析与可视化Guides data acquisition, cleaning, statistical analysis, and visualization for reproducible reporting. Use when building a data-driven story, map, or public methodology.
- 数字内容归档与知识图谱Designs content archive schemas with AI-assisted categorization, entity extraction, and knowledge-graph construction. Use when building a searchable archive or linking media entities.
- 编辑部选题流程管理Provides templates for story tracking, editorial calendars, and assignment handoffs in a newsroom. Use when managing deadlines or coordinating reporters and editors.
- 新闻事实核查流程Structures claim extraction, evidence gathering, and rating decisions for pre-publication fact-checking. Use when verifying claims before publishing or building a dedicated fact-check.
- 政府信息公开申请Guides drafting, tracking, and appealing US federal and state public-records requests. Use when filing a FOIA-style request or checking exemptions and deadlines.
- 采访准备与录音合规Prepares interview questions by type and explains recording-consent law by US state. Use before an interview to plan questions or confirm consent requirements.
- 采访录音转写整理Guides recording setup, automated transcription with speaker diarization, and quote extraction. Use when processing interview audio or generating timestamped transcripts.
- 报道法律风险审查Flags defamation, privacy-tort, and prior-restraint risk in a draft before legal review or publication. Use before publishing a sensitive story naming identifiable people.
- 新闻写作风格规范Applies AP Style rules for numbers, titles, attribution, and ledes to news copy. Use when editing drafts, writing headlines, or tightening copy for publication.
- 新闻图片元数据嵌入Embeds IPTC, EXIF, and XMP metadata including caption, credit, alt text, AI-source labeling, and GPS stripping. Use before publishing or distributing press photos.
- 民调方法与可信度评估Evaluates poll methodology, margin of error, weighting, and question wording before citing or reporting results. Use before citing, commissioning, or writing about a poll.
- 采编任务联系人记录核对Reconciles task, contact, and interaction records against canonical entries before writing changes. Use when new intake names a person, assignment, or status change.
- 社交媒体舆情监测Guides multi-platform monitoring, account authenticity checks, and coordinated-behavior detection for OSINT. Use when tracking viral spread or vetting suspicious accounts.
- 信源与内容真实性核验Applies the SIFT method to verify sources, images, video, and documents with a durable evidence trail. Use before relying on uncertain material in reporting.
- 新闻选题投稿策划Provides pitch templates for daily news, features, investigations, and op-eds across outlets. Use when pitching editors, writing query letters, or shaping a story angle.
- 网页内容存档取证Guides preserving web pages via Wayback Machine, Archive.today, and the CDX API for evidence. Use when a page may disappear or you need to reach a dead link.
常见问题
内容分析编码方法能做什么?
Designs codebooks, sampling plans, and intercoder-reliability checks for systematic content analysis. Use when measuring patterns in media coverage or validating an LLM classifier。
如何安装内容分析编码方法?
在科搭 Codar 左侧点击「插件」,在「组件」页签搜索「内容分析编码方法」并安装;在 Claude Code 或 Codex 中可以发送本页的安装提示词,或下载 Skill 包后解压到 ~/.claude/skills 或 ~/.codex/skills。
内容分析编码方法需要哪些权限?
该科研 Skill没有声明额外权限;执行具体操作时仍由 Agent 按自身规则请求确认。
内容分析编码方法经过官方审核吗?
是。SpeedSkills 收录的每个版本都会记录内容摘要与权限声明,当前版本已通过安全审核。