科研 Skill

量表信效度验证

survey-instrument-validation

Runs reliability, factor-structure and invariance checks on a questionnaire or PRO. Use when a scale needs psychometric validation before or after it is used.

发布者 Codar安全审核通过v0.1.0更新于 2026-11-18

SKILL.md

Survey Instrument / PRO Validation Skill

Produces and audits the psychometric evidence a new or adapted questionnaire, scale, or patient-reported outcome (PRO) measure needs before it is used in research or reported in a manuscript: internal consistency, structural validity (factor structure), construct validity, test-retest reliability, and, for cross-cultural adaptations, measurement invariance. This is the instrument-development counterpart to analyze-stats (which runs the substantive analysis once a validated instrument already exists) and to define-variables (which maps definitions to database columns, not to a multi-item scale's latent structure).

When to use

  • A new questionnaire or scale is being developed and needs reliability/validity evidence before its first use.
  • An existing instrument is being translated or adapted to a new language, country, or population and needs a validation (not just translation) study.
  • A manuscript uses a scale and a reviewer will ask for its psychometric properties in this sample (reliability is sample-dependent — a published alpha from the original validation paper is not evidence for a new population).
  • Choosing between competing instruments that claim to measure the same construct.

When NOT to use

  • The instrument is already validated for this exact population/language and the study only uses its scored output as one more variable in a model → analyze-stats.
  • Open-ended/free-text responses → qualitative-research.
  • Sample size planning for the validation study itself → calc-sample-size (ask for this skill's minimum-N rules of thumb as an input, e.g. "N ≥ 10 per item for EFA" is a starting heuristic, not a substitute for a proper power calculation when one is feasible).

Prerequisites

  • Python 3.10+ with pandas, numpy, scipy, factor_analyzer, semopy (or R with psych, lavaan, mirt) depending on the analysis requested.
  • Item-level response data (not only the summed/scored total) — structural and reliability analyses need every item's responses, and a dataset containing only the final score cannot be validated.
  • A stated conceptual model: what construct(s) the instrument is meant to measure, and the hypothesized subscale structure if there is more than one dimension, decided before running exploratory factor analysis — EFA explores structure within a theory, it does not replace one.
  • For PRO development: FDA (if a US regulatory submission is the goal) or equivalent guidance expects evidence of content validity from the target population (cognitive debriefing / think-aloud interviews) in addition to the quantitative properties this skill covers — flag this as a prerequisite step, not something the numeric analysis alone can satisfy.

Workflow

Phase 1 — Frame the validation question

  1. New instrument, translation/adaptation, or re-validation in a new population? Each has a different minimum evidence bar (a translation needs cross-language measurement invariance; a brand-new instrument needs content validity evidence this skill does not generate).
  2. Confirm the response scale (binary, Likert, visual analogue) and scoring rule (simple sum, weighted sum, IRT-scored) as specified by the instrument's developer or protocol.
  3. State the hypothesized dimensionality: unidimensional, or multi-subscale with named subscales and their item assignments.

Phase 2 — Item-level and internal-consistency checks

import pandas as pd
from pingouin import cronbach_alpha

items = df[item_columns]
alpha, ci = cronbach_alpha(data=items)
print(f"Cronbach's alpha = {alpha:.3f}, 95% CI {ci}")
  1. Inspect item-level distributions first: floor/ceiling effects (>15-20% at the scale extreme), missingness per item, and item-total correlations (an item correlating <0.20 with the corrected total is a candidate for removal or revision, not an automatic drop — check content relevance before removing).
  2. Report Cronbach's alpha (for a reflective, internally consistent scale) and flag that a high alpha with many items can be an artifact of item count, not true homogeneity (McDonald's omega is a less length-dependent alternative — report it alongside alpha when the factor structure is known).
  3. For a multi-subscale instrument, compute alpha/omega per subscale, not only for the total score.

Phase 3 — Structural validity (factor analysis)

from factor_analyzer import FactorAnalyzer, calculate_kmo, calculate_bartlett_sphericity

kmo_all, kmo_model = calculate_kmo(items)
chi2, p = calculate_bartlett_sphericity(items)
fa = FactorAnalyzer(n_factors=hypothesized_k, rotation="oblimin")
fa.fit(items)
print(fa.loadings_)
  1. Check sampling adequacy (KMO ≥ 0.6, Bartlett's test p < .05) before trusting any factor solution — if these fail, say the data does not support a factor analysis rather than reporting one anyway.
  2. New instrument / no prior factor structure → exploratory factor analysis (EFA) on one random half of the sample (or an independent sample), then confirm the resulting structure with confirmatory factor analysis (CFA) on the other half — fitting a CFA on the exact same data an EFA was run on reports an inflated fit index and is circular.
  3. Prior factor structure exists (translation, re-validation) → go straight to CFA testing the published structure; report standard fit indices (CFI/TLI ≥ 0.90-0.95, RMSEA ≤ 0.06-0.08, SRMR ≤ 0.08 — cite the cutoffs source used) and item factor loadings (≥0.40 as a common, not universal, floor).
  4. If CFA fit is poor, do not silently respecify the model to improve fit without stating every post-hoc modification and its substantive justification — undisclosed model modification indices chasing fit is a well-known source of non-replicable factor structures.

Phase 4 — Construct validity (convergent, discriminant, known-groups)

  1. Convergent validity: correlate the new instrument's score(s) with an existing measure of the same or a related construct; report the correlation and its CI, and the a priori expected direction/magnitude.
  2. Discriminant validity: correlate with a measure of a conceptually distinct construct; expect a weaker correlation than the convergent comparison.
  3. Known-groups validity: test whether the instrument distinguishes groups it should theoretically distinguish (e.g., diagnosed vs non-diagnosed) using the appropriate test from analyze-stats's decision tree — route the test selection there rather than re-deriving it.

Phase 5 — Reliability over time and across raters

  1. Test-retest reliability: administer to the same respondents at an interval short enough that the true construct should not have changed, but long enough to avoid recall of prior answers (instrument- and construct-specific; state and justify the chosen interval). Report the intraclass correlation coefficient (ICC, two-way mixed/random per the reliability question being asked — route the ICC-form choice to analyze-stats's agreement guide) for continuous scores, weighted kappa for ordinal categories.
  2. Inter-rater reliability (observer/clinician-rated instruments only): ICC or weighted kappa between independent raters scoring the same encounters.

Phase 6 — Measurement invariance (translations and cross-group comparisons)

For comparing scores across languages, countries, or demographic groups (not just within one sample):

  1. Fit a sequence of increasingly constrained multi-group CFA models: configural (same factor structure, free loadings) → metric (equal loadings) → scalar (equal loadings and intercepts).
  2. Compare nested models with a fit-index change criterion (e.g., ΔCFI ≤ -0.010 with ΔRMSEA ≤ 0.015 supporting invariance — cite the criterion used) rather than only a chi-square difference test, which is oversensitive in large samples.
  3. Scalar invariance is required before comparing mean scores across groups — comparing raw scores across languages/cultures without at least scalar invariance risks attributing a measurement artifact to a real group difference. State the invariance level actually achieved and limit cross-group claims accordingly.

Standard Output

## Instrument Validation Report
Instrument: name, N items, hypothesized structure
Sample: N, population, language

### Internal consistency
- Cronbach's alpha (McDonald's omega): total and per subscale

### Structural validity
- KMO / Bartlett: pass/fail
- EFA/CFA: n factors, fit indices, items with loading < 0.40

### Construct validity
- Convergent r (vs ...): ... | Discriminant r (vs ...): ... | Known-groups test: ...

### Reliability over time/raters
- Test-retest ICC/kappa (interval): ... | Inter-rater ICC/kappa: ...

### Measurement invariance (if cross-group)
- Highest level achieved: configural | metric | scalar

### Recommendation
- Fit for purpose / needs revision / insufficient evidence, with the specific gap named

Quality checks

  • Never report alpha/omega from a published source as if it were evidence for the current sample — every reliability figure in the report is computed from the data in hand.
  • EFA and the CFA that confirms it never run on the identical data without an explicit disclosure that this is exploratory-only and needs independent confirmation.
  • Every post-hoc CFA model modification (added covariance, dropped item) is logged with its modification index and substantive rationale, not silently applied.
  • Cross-group score comparisons are gated on the invariance level actually demonstrated, not assumed.

Limitations

  • Psychometric properties are sample- and context-dependent; a figure reported here does not transfer to a different population, language, or administration mode (paper vs electronic, self- vs interviewer-administered) without re-validation.
  • This skill does not establish content validity (whether the items capture the construct's full content domain) — that requires qualitative work (expert panels, cognitive interviews, often via qualitative-research) that must happen before or alongside the quantitative analysis here.
  • Small samples (rule-of-thumb minimums such as N ≥ 10 per item for EFA, N ≥ 200 for a stable CFA) produce unstable factor solutions; this skill will flag an under-powered sample rather than report a solution with false confidence, but cannot make a small sample larger.
  • For formal regulatory PRO qualification (e.g., FDA PRO guidance), this skill's quantitative outputs are necessary but not sufficient; the full qualification dossier needs the qualitative content-validity work and a regulatory strategy outside this skill's scope.

示例提示词

帮我验证这份新翻译的生活质量量表的信度和因子结构。

所属插件

临床医学与公共卫生面向临床与公卫研究:研究设计与方案、样本量、队列与 Meta 分析、报告规范核查、医学影像 AI 评估、流行病建模与卫生经济学评价、医学论文写作投稿。

安装该插件会一并获得同领域的 58 个 Skill / MCP。

同插件的其他科研 Skill

常见问题

量表信效度验证能做什么?

Runs reliability, factor-structure and invariance checks on a questionnaire or PRO. Use when a scale needs psychometric validation before or after it is used。

如何安装量表信效度验证?

在科搭 Codar 左侧点击「插件」,在「组件」页签搜索「量表信效度验证」并安装;在 Claude Code 或 Codex 中可以发送本页的安装提示词,或下载 Skill 包后解压到 ~/.claude/skills 或 ~/.codex/skills。

量表信效度验证需要哪些权限?

该科研 Skill没有声明额外权限;执行具体操作时仍由 Agent 按自身规则请求确认。

量表信效度验证经过官方审核吗?

是。SpeedSkills 收录的每个版本都会记录内容摘要与权限声明,当前版本已通过安全审核。