科研 Skill

量表测量质量把关

alterlab-ssci-measurement-gate

Checks that a scale’s reliability, validity, and measurement invariance are established before trusting or comparing it across groups. Use when adopting or reporting a survey instrument.

发布者 Codar安全审核通过v0.1.0更新于 2026-11-18

SKILL.md

Measurement Gate — Reliability Is Not Validity

Skill type: DISCIPLINE-ENFORCING. Before any hypothesis test consumes a scale, this gate checks that the scale measures what it claims, reliably, and comparably across groups. It does not fit factor models itself — it requires the right evidence and routes execution to the psychometrics skill.

The Core Rule

A RELIABLE MEASURE OF THE WRONG CONSTRUCT IS STILL WRONG. RELIABILITY ≠ VALIDITY.

High internal consistency only means the items move together; it says nothing about whether they capture the intended construct. Evidence for validity (content, criterion, construct) is separate and mandatory. And the usual reliability number — Cronbach alpha — rests on an assumption (tau-equivalence: equal item loadings) that real scales rarely meet, so alpha is a lower bound that can under- or (with correlated errors) over-state reliability.

When to Use This Skill

  • "Is my scale / survey instrument valid?"
  • "My Cronbach alpha is 0.78 — is the scale good to use?" (← alpha alone is not enough)
  • "I built a new 6-item measure of burnout; how do I show it works?"
  • "I want to compare a latent trait (e.g. trust) across countries/groups."

Does NOT Trigger

The request is really about…Route toWhy not this skill
Writing / wording the questionnaire items, response scalesalterlab-survey-designInstrument construction, upstream of validation.
Running the CFA / SEM / IRT / invariance modelalterlab-sem-psychometricsThis gate requires CFA; that skill fits it.
Basic descriptive/inferential stats executionalterlab-statistical-analysisComputation, not measurement discipline.
Which statistical test to run on a measured outcomealterlab-test-selection-guardTest choice, not measurement quality.
Sampling frame / how many respondentsalterlab-ssci-sampling-gateSampling, not measurement.

Hand-off rule. When the request matches a row above, open your answer by naming that skill by its exact id (for example alterlab-survey-design) and saying in one line why it fits better, then stop applying this skill's procedure. Name it even when you also need to ask for missing input. If only part of the request is out of scope, name the skill for that part and do only the in-scope part. A brief general pointer is fine; the named skill does the work.

Reliability: report alpha AND omega

Cronbach's alpha equals reliability only under essential tau-equivalence (all items load equally on one factor) with uncorrelated errors; when loadings differ it is a lower bound, and correlated errors can push it above the true value (Sijtsma, 2009, Psychometrika). McDonald's omega relaxes tau-equivalence by weighting items by their factor loadings and should be reported alongside alpha (McNeish, 2018, Psychological Methods, "Thanks coefficient alpha, we'll take it from here"). In practice the alpha-vs-omega gap for well-constructed unidimensional scales is often small (in one cognitive test battery, omega exceeded alpha by 4.5% on average — Warne, 2025, blog analysis), so the discipline is: report both, plus item-total correlations and a dimensionality check — not to fetishize a single number, but to show the estimate is not resting on an unexamined assumption.

Rules this gate enforces:

  1. Alpha alone is insufficient. Ask for omega; if only alpha is available, state the tau-equivalence caveat explicitly.
  2. Reliability presupposes dimensionality. A high alpha on a multidimensional set is meaningless — check the factor structure (CFA, alterlab-sem-psychometrics) first.
  3. Item diagnostics. Item-total correlations and (for two halves) Spearman-Brown; drop or revise items that do not cohere — but pre-specify such decisions, do not fish.

Validity: four kinds, none optional

ValidityQuestionEvidence
ContentDo items cover the construct's domain?expert review, blueprint against the definition
CriterionDoes it predict/correlate with a gold standard?concurrent & predictive correlations
ConvergentDoes it correlate with related measures?AVE, correlations with kindred scales
DiscriminantIs it distinct from other constructs?heterotrait-monotrait ratio, cross-loadings

Construct validity is established with a confirmatory factor analysis — route the fit to alterlab-sem-psychometrics (CFI/TLI, RMSEA, SRMR against Hu & Bentler cutoffs).

Measurement invariance before group comparison

Comparing a latent construct across groups (countries, waves, conditions) is only valid if the measure means the same thing in each group. Require a multi-group CFA invariance sequence — configural → metric → scalar — before interpreting group differences in means. Without at least metric/scalar invariance, a "difference" may be a measurement artifact, not a real effect. Route the test to alterlab-sem-psychometrics.

The Design Passport (hand-off)

Append to each construct in the Passport: reliability (alpha + omega + caveat), validity_evidence (which of the four kinds are shown), dimensionality (CFA fit), and invariance (level established) before any group comparison downstream.

Self-Check Before Advancing

  • Is each construct explicitly defined and operationalized to its items?
  • Is reliability shown with omega (or alpha with the tau-equivalence caveat stated)?
  • Is at least one form of validity evidence present — not just reliability?
  • Was dimensionality confirmed (CFA) before trusting a single reliability coefficient?
  • Is measurement invariance established before comparing the construct across groups?

References

  • references/reliability_and_validity.md — alpha vs omega math, the four validities, the invariance sequence, and reporting templates. Loaded on demand.

Part of the AlterLab Academic Skills suite.

示例提示词

帮我检查这份量表的信度效度报告是否只靠Cronbach alpha撑场面。

所属插件

社会学、政治学与城市研究面向社会科学研究:问卷与访谈设计、质性编码、文本即数据、社会网络、ABM、比较政治与选举数据、RCT 与 DID、城市与区域分析。

安装该插件会一并获得同领域的 58 个 Skill / MCP。

同插件的其他科研 Skill

常见问题

量表测量质量把关能做什么?

Checks that a scale’s reliability, validity, and measurement invariance are established before trusting or comparing it across groups. Use when adopting or reporting a survey instrument。

如何安装量表测量质量把关?

在科搭 Codar 左侧点击「插件」,在「组件」页签搜索「量表测量质量把关」并安装;在 Claude Code 或 Codex 中可以发送本页的安装提示词,或下载 Skill 包后解压到 ~/.claude/skills 或 ~/.codex/skills。

量表测量质量把关需要哪些权限?

该科研 Skill没有声明额外权限;执行具体操作时仍由 Agent 按自身规则请求确认。

量表测量质量把关经过官方审核吗?

是。SpeedSkills 收录的每个版本都会记录内容摘要与权限声明,当前版本已通过安全审核。