科研 Skill

民调方法与可信度评估

public-opinion-polling-analysis

Evaluates poll methodology, margin of error, weighting, and question wording before citing or reporting results. Use before citing, commissioning, or writing about a poll.

发布者 Codar安全审核通过v0.1.0更新于 2026-11-18

SKILL.md

Public opinion polling and survey analysis

Methods for reading, critiquing, and reporting on public opinion polls and surveys — the specific research-methods layer that general data-journalism doesn't cover: sampling design, margin of error, weighting, mode effects, and question-wording bias.

Purpose

A poll's topline number is the least informative part of the poll. The methodology, who was sampled, how, with what response rate, weighted how, asked which exact question, in what order, determines whether the topline means anything. This skill gives a reporter or communication researcher a repeatable checklist for evaluating a poll before citing it, and for designing or commissioning one.

When to use

  • Deciding whether to cite a poll released by a campaign, advocacy group, or pollster.
  • Writing a story about poll results and need to report the margin of error and methodology correctly.
  • Designing or commissioning an original survey for a story or research project.
  • Comparing two polls on the same race or question that show different results (a "polling average" or "poll of polls" problem).
  • Reviewing a communication-research survey instrument for construct validity before fielding it.

Step 1: Find the methodology, not just the topline

Never cite a number without finding (or requesting) the methodology statement. At minimum, it must disclose:

  • Sponsor and pollster — who paid for it, who conducted it (an internal campaign poll and an independent media poll carry very different credibility).
  • Population and sample — likely voters, registered voters, adults; the distinction matters enormously close to an election (likely-voter screens are themselves a modeling choice that can shift results several points).
  • Sample size (n) and the margin of error it implies.
  • Mode — live phone, IVR/robopoll, online panel, text-to-web, mixed mode.
  • Field dates — when it was conducted; a poll fielded before a major news event is stale for assessing reaction to that event.
  • Weighting variables — age, gender, race, education, party ID, geography; which ones were used and to what population benchmarks (usually Census American Community Survey or exit-poll composites).
  • Response rate, where available (increasingly rare to disclose, but a AAPOR-compliant release should at least note it or explain its absence).
  • Exact question wording and order, including any preceding questions that could prime the response.

The American Association for Public Opinion Research (AAPOR, aapor.org) Transparency Initiative sets the standard for what should be disclosed; treat a poll that won't disclose the above as lower-credibility by default, and say so if you report on it anyway.

Step 2: Margin of error — what it does and doesn't cover

For a simple random sample, the margin of error (MOE) at 95% confidence is approximately:

MOE ≈ 1.96 × sqrt( p(1-p) / n )

where p is the proportion (use 0.5 for the most conservative, widest MOE) and n is the sample size. For a quick worst-case estimate with p = 0.5:

MOE ≈ 0.98 / sqrt(n)

At n=1,000, MOE ≈ ±3.1 points. At n=400 (a typical subgroup crosstab), MOE blows up to ≈ ±4.9 points — subgroup crosstabs almost always carry a much wider MOE than the topline, and most horse-race coverage ignores this when citing a subgroup breakdown.

What MOE does not capture (and why two "statistically tied" polls can still diverge):

  • Non-response bias: who refuses to answer, and whether refusal correlates with the thing you're measuring (a classic 2016/2020 US election problem — differential non-response by education and partisanship).
  • Mode effects: live-caller, IVR, and online-panel polls on the same question can differ by several points for reasons unrelated to sample size.
  • House effects: a given pollster can show a persistent lean relative to the polling average, tracked by aggregators (see Tools below).
  • Weighting-model uncertainty: if the likely-voter model or weighting targets are wrong, no amount of sample size fixes it.

When two polls differ by less than the combined margin of error, they are not necessarily in tension, report this explicitly rather than writing "which poll is right."

Step 3: Question wording and order effects

Small wording changes move numbers. Checklist for evaluating or writing questions:

  • Loaded or leading language: "Do you support the common-sense policy of X" versus "Do you support X" — the former primes agreement.
  • Balanced vs. one-sided framing: a question that states only one side's argument before asking for an opinion will shift results toward that side.
  • Order effects: an earlier question can prime a later one (asking about crime before asking about a candidate who campaigns on crime).
  • Forced choice vs. don't know/no opinion offered: suppressing a "don't know" option inflates apparent certainty.
  • Double-barreled questions: "Do you support stricter gun laws and more mental health funding" asks two things at once and can't be cleanly coded.

When reporting on a poll commissioned by an advocacy group or campaign, request and read the exact question wording before citing the number, these polls are the most likely to use framing that favors the sponsor.

Step 4: Weighting and representativeness

Raw survey respondents rarely match the target population's demographics. Weighting (raking/iterative proportional fitting is the common method) adjusts each respondent's influence so the weighted sample matches known population totals (Census ACS for demographics, voter file data or past turnout models for likely-voter polls).

Red flags:

  • Weighting on too few variables (age and gender only, ignoring education, which has been the single largest driver of polling error in recent US election cycles).
  • Extreme weights on a small number of respondents (a poorly weighted poll can have its result driven by a handful of heavily-upweighted respondents); ask for the design effect or effective sample size if available.
  • No disclosure of weighting targets at all.

Tools and data sources

Tool / sourceUse
R survey package or Python samplicsCompute weighted estimates, design effects, and correct standard errors for complex survey designs
538 / Silver Bulletin pollster ratingsHistorical accuracy and house-effect tracking by pollster
AAPOR Transparency Initiative (aapor.org)Standard for methodology disclosure; member pollsters commit to releasing the items in Step 1
Roper Center for Public Opinion Research (ropercenter.cornell.edu)Archive of historical US poll questions and results, useful for trend comparisons
Pew Research Center methodology reports (pewresearch.org/methods)Published methodology and mode-effect studies, a model for how to write up your own
General Social Survey (GSS) (gss.norc.org)Long-running US benchmark survey, useful for validating question wording against a trusted baseline
Qualtrics, SurveyMonkey, Google FormsSurvey fielding tools for original research; none of these handle probability-based sampling or weighting on their own

Reporting template

## Poll: [headline topic]

**Sponsor/pollster:** [who paid, who conducted]
**Population:** [likely voters / registered voters / adults], n=[sample size]
**Mode:** [phone/online/mixed], field dates [start–end]
**Margin of error:** ±[X] points at 95% confidence (topline);
  subgroup MOEs are wider, state separately if citing a crosstab
**Weighting:** [variables used, benchmark source]
**Exact question asked:** "[verbatim text]"

**What the number shows:** [plain-language finding]
**What it doesn't show:** [differences within MOE, non-disclosed items,
  limitations]
**How it compares:** [other recent polls on the same question, and whether
  differences exceed combined MOE]

Quality checks before publishing a poll story

  • Methodology statement located and read, not just the press release.
  • Margin of error reported for the specific number being cited (topline vs. subgroup).
  • Sponsor and any financial interest in the result disclosed to readers.
  • Exact question wording checked for leading or loaded language.
  • If comparing polls, confirmed whether the difference exceeds combined MOE before writing language like "surging" or "collapsing."
  • Field dates checked against any news events that could explain movement.
  • If the poll has under ~5% response rate or undisclosed response rate (common for online-panel and robopoll surveys), this limitation is noted rather than presented as equivalent to a probability sample.

Prerequisites

  • Basic statistics (confidence intervals, proportions) is required to interpret margin of error and weighting discussions correctly.
  • Requires the pollster's methodology disclosure; if a pollster won't release it, that absence is itself a reportable finding, not a reason to skip the check.

Limitations

  • This skill does not replace a trained survey statistician for designing a new probability sample, weighting scheme, or likely-voter model; it is a reporting and critique checklist, not a sampling-design tool.
  • Polling methodology and best practice shift (new modes, declining phone response rates, panel-based online sampling becoming dominant); verify current AAPOR guidance rather than treating this file as the permanent state of the art.
  • Margin-of-error formulas here assume a simple random sample; real polls use complex designs (stratification, clustering, weighting) where the true margin of error is usually larger than the naive formula suggests, this is why some pollsters report a "design effect"-adjusted MOE.

Related skills

  • data-journalism, Statistical analysis, denominators, and chart honesty once poll data is in hand
  • fact-check-workflow, Verify specific factual claims a campaign or group makes about a poll
  • source-verification, Check a pollster's track record and independence before trusting their numbers

示例提示词

帮我看看这份民调的抽样方法、误差范围和题目措辞是否可信。

所属插件

新闻传播学面向新闻传播研究与调查报道:事实核查与信源核验、数据新闻、信息公开申请、采访准备与转录、编辑流程与新闻文风、社交媒体情报、图片元数据、网页与数字存档、危机传播。

安装该插件会一并获得同领域的 19 个 Skill / MCP。

同插件的其他科研 Skill

常见问题

民调方法与可信度评估能做什么?

Evaluates poll methodology, margin of error, weighting, and question wording before citing or reporting results. Use before citing, commissioning, or writing about a poll。

如何安装民调方法与可信度评估?

在科搭 Codar 左侧点击「插件」,在「组件」页签搜索「民调方法与可信度评估」并安装;在 Claude Code 或 Codex 中可以发送本页的安装提示词,或下载 Skill 包后解压到 ~/.claude/skills 或 ~/.codex/skills。

民调方法与可信度评估需要哪些权限?

该科研 Skill没有声明额外权限;执行具体操作时仍由 Agent 按自身规则请求确认。

民调方法与可信度评估经过官方审核吗?

是。SpeedSkills 收录的每个版本都会记录内容摘要与权限声明,当前版本已通过安全审核。