科研 Skill
量表信效度验证
survey-instrument-validation
Runs reliability, factor-structure and invariance checks on a questionnaire or PRO. Use when a scale needs psychometric validation before or after it is used.
SKILL.md
Survey Instrument / PRO Validation Skill
Produces and audits the psychometric evidence a new or adapted questionnaire, scale, or
patient-reported outcome (PRO) measure needs before it is used in research or reported in a
manuscript: internal consistency, structural validity (factor structure), construct validity,
test-retest reliability, and, for cross-cultural adaptations, measurement invariance. This is the
instrument-development counterpart to analyze-stats (which runs the substantive analysis once a
validated instrument already exists) and to define-variables (which maps definitions to database
columns, not to a multi-item scale's latent structure).
When to use
- A new questionnaire or scale is being developed and needs reliability/validity evidence before its first use.
- An existing instrument is being translated or adapted to a new language, country, or population and needs a validation (not just translation) study.
- A manuscript uses a scale and a reviewer will ask for its psychometric properties in this sample (reliability is sample-dependent — a published alpha from the original validation paper is not evidence for a new population).
- Choosing between competing instruments that claim to measure the same construct.
When NOT to use
- The instrument is already validated for this exact population/language and the study only uses
its scored output as one more variable in a model →
analyze-stats. - Open-ended/free-text responses →
qualitative-research. - Sample size planning for the validation study itself →
calc-sample-size(ask for this skill's minimum-N rules of thumb as an input, e.g. "N ≥ 10 per item for EFA" is a starting heuristic, not a substitute for a proper power calculation when one is feasible).
Prerequisites
- Python 3.10+ with
pandas,numpy,scipy,factor_analyzer,semopy(or R withpsych,lavaan,mirt) depending on the analysis requested. - Item-level response data (not only the summed/scored total) — structural and reliability analyses need every item's responses, and a dataset containing only the final score cannot be validated.
- A stated conceptual model: what construct(s) the instrument is meant to measure, and the hypothesized subscale structure if there is more than one dimension, decided before running exploratory factor analysis — EFA explores structure within a theory, it does not replace one.
- For PRO development: FDA (if a US regulatory submission is the goal) or equivalent guidance expects evidence of content validity from the target population (cognitive debriefing / think-aloud interviews) in addition to the quantitative properties this skill covers — flag this as a prerequisite step, not something the numeric analysis alone can satisfy.
Workflow
Phase 1 — Frame the validation question
- New instrument, translation/adaptation, or re-validation in a new population? Each has a different minimum evidence bar (a translation needs cross-language measurement invariance; a brand-new instrument needs content validity evidence this skill does not generate).
- Confirm the response scale (binary, Likert, visual analogue) and scoring rule (simple sum, weighted sum, IRT-scored) as specified by the instrument's developer or protocol.
- State the hypothesized dimensionality: unidimensional, or multi-subscale with named subscales and their item assignments.
Phase 2 — Item-level and internal-consistency checks
import pandas as pd
from pingouin import cronbach_alpha
items = df[item_columns]
alpha, ci = cronbach_alpha(data=items)
print(f"Cronbach's alpha = {alpha:.3f}, 95% CI {ci}")
- Inspect item-level distributions first: floor/ceiling effects (>15-20% at the scale extreme), missingness per item, and item-total correlations (an item correlating <0.20 with the corrected total is a candidate for removal or revision, not an automatic drop — check content relevance before removing).
- Report Cronbach's alpha (for a reflective, internally consistent scale) and flag that a high alpha with many items can be an artifact of item count, not true homogeneity (McDonald's omega is a less length-dependent alternative — report it alongside alpha when the factor structure is known).
- For a multi-subscale instrument, compute alpha/omega per subscale, not only for the total score.
Phase 3 — Structural validity (factor analysis)
from factor_analyzer import FactorAnalyzer, calculate_kmo, calculate_bartlett_sphericity
kmo_all, kmo_model = calculate_kmo(items)
chi2, p = calculate_bartlett_sphericity(items)
fa = FactorAnalyzer(n_factors=hypothesized_k, rotation="oblimin")
fa.fit(items)
print(fa.loadings_)
- Check sampling adequacy (KMO ≥ 0.6, Bartlett's test p < .05) before trusting any factor solution — if these fail, say the data does not support a factor analysis rather than reporting one anyway.
- New instrument / no prior factor structure → exploratory factor analysis (EFA) on one random half of the sample (or an independent sample), then confirm the resulting structure with confirmatory factor analysis (CFA) on the other half — fitting a CFA on the exact same data an EFA was run on reports an inflated fit index and is circular.
- Prior factor structure exists (translation, re-validation) → go straight to CFA testing the published structure; report standard fit indices (CFI/TLI ≥ 0.90-0.95, RMSEA ≤ 0.06-0.08, SRMR ≤ 0.08 — cite the cutoffs source used) and item factor loadings (≥0.40 as a common, not universal, floor).
- If CFA fit is poor, do not silently respecify the model to improve fit without stating every post-hoc modification and its substantive justification — undisclosed model modification indices chasing fit is a well-known source of non-replicable factor structures.
Phase 4 — Construct validity (convergent, discriminant, known-groups)
- Convergent validity: correlate the new instrument's score(s) with an existing measure of the same or a related construct; report the correlation and its CI, and the a priori expected direction/magnitude.
- Discriminant validity: correlate with a measure of a conceptually distinct construct; expect a weaker correlation than the convergent comparison.
- Known-groups validity: test whether the instrument distinguishes groups it should
theoretically distinguish (e.g., diagnosed vs non-diagnosed) using the appropriate test from
analyze-stats's decision tree — route the test selection there rather than re-deriving it.
Phase 5 — Reliability over time and across raters
- Test-retest reliability: administer to the same respondents at an interval short enough
that the true construct should not have changed, but long enough to avoid recall of prior
answers (instrument- and construct-specific; state and justify the chosen interval). Report the
intraclass correlation coefficient (ICC, two-way mixed/random per the reliability question
being asked — route the ICC-form choice to
analyze-stats's agreement guide) for continuous scores, weighted kappa for ordinal categories. - Inter-rater reliability (observer/clinician-rated instruments only): ICC or weighted kappa between independent raters scoring the same encounters.
Phase 6 — Measurement invariance (translations and cross-group comparisons)
For comparing scores across languages, countries, or demographic groups (not just within one sample):
- Fit a sequence of increasingly constrained multi-group CFA models: configural (same factor structure, free loadings) → metric (equal loadings) → scalar (equal loadings and intercepts).
- Compare nested models with a fit-index change criterion (e.g., ΔCFI ≤ -0.010 with ΔRMSEA ≤ 0.015 supporting invariance — cite the criterion used) rather than only a chi-square difference test, which is oversensitive in large samples.
- Scalar invariance is required before comparing mean scores across groups — comparing raw scores across languages/cultures without at least scalar invariance risks attributing a measurement artifact to a real group difference. State the invariance level actually achieved and limit cross-group claims accordingly.
Standard Output
## Instrument Validation Report
Instrument: name, N items, hypothesized structure
Sample: N, population, language
### Internal consistency
- Cronbach's alpha (McDonald's omega): total and per subscale
### Structural validity
- KMO / Bartlett: pass/fail
- EFA/CFA: n factors, fit indices, items with loading < 0.40
### Construct validity
- Convergent r (vs ...): ... | Discriminant r (vs ...): ... | Known-groups test: ...
### Reliability over time/raters
- Test-retest ICC/kappa (interval): ... | Inter-rater ICC/kappa: ...
### Measurement invariance (if cross-group)
- Highest level achieved: configural | metric | scalar
### Recommendation
- Fit for purpose / needs revision / insufficient evidence, with the specific gap named
Quality checks
- Never report alpha/omega from a published source as if it were evidence for the current sample — every reliability figure in the report is computed from the data in hand.
- EFA and the CFA that confirms it never run on the identical data without an explicit disclosure that this is exploratory-only and needs independent confirmation.
- Every post-hoc CFA model modification (added covariance, dropped item) is logged with its modification index and substantive rationale, not silently applied.
- Cross-group score comparisons are gated on the invariance level actually demonstrated, not assumed.
Limitations
- Psychometric properties are sample- and context-dependent; a figure reported here does not transfer to a different population, language, or administration mode (paper vs electronic, self- vs interviewer-administered) without re-validation.
- This skill does not establish content validity (whether the items capture the construct's full
content domain) — that requires qualitative work (expert panels, cognitive interviews, often via
qualitative-research) that must happen before or alongside the quantitative analysis here. - Small samples (rule-of-thumb minimums such as N ≥ 10 per item for EFA, N ≥ 200 for a stable CFA) produce unstable factor solutions; this skill will flag an under-powered sample rather than report a solution with false confidence, but cannot make a small sample larger.
- For formal regulatory PRO qualification (e.g., FDA PRO guidance), this skill's quantitative outputs are necessary but not sufficient; the full qualification dossier needs the qualitative content-validity work and a regulatory strategy outside this skill's scope.
示例提示词
帮我验证这份新翻译的生活质量量表的信度和因子结构。
所属插件
安装该插件会一并获得同领域的 58 个 Skill / MCP。
同插件的其他科研 Skill
- 医学AI论文检索优化Optimizes a medical AI paper’s title, abstract and metadata for AI search engines. Use when a paper should be findable and citable by RAG tools and AI search.
- 临床数据统计分析与建模Generates reproducible Python or R code for clinical statistics and tables. Use when collected data needs Table 1, regression, survival or agreement analysis.
- 作者发表记录与策略分析Analyzes a PubMed author record to classify study types and authorship patterns. Use when planning a publication strategy from a researcher’s track record.
- 批量队列暴露结局分析Generates one cohort-analysis script per exposure-outcome pair from one template. Use when a validated method must run across many variable combinations at once.
- 样本量估算与检验效能Walks a decision tree to the right test and computes sample size with IRB text. Use when planning how many patients a study needs before data collection.
- 因果推断方法设计Runs DAG-based, instrumental-variable, difference-in-differences or RD analyses. Use when an association in observational data must support a causal claim.
- 报告规范逐条核查Audits a manuscript item by item against 49 reporting and risk-of-bias checklists. Use when checking a draft against STROBE, CONSORT, TRIPOD or similar guidelines.
- 临床数据清洗与画像Profiles a clinical dataset and generates cleaning code in staged, approved steps. Use when a CSV or Excel dataset needs missing-value and outlier checks before analysis.
- 跨国队列数据协调对比Harmonizes variables and runs parallel weighted analyses across national surveys like KNHANES and NHANES. Use when comparing one exposure-outcome link across countries.
- 变量定义与截断值溯源Maps exposure, outcome and covariate definitions to a cited guideline and DB columns. Use when variable cutoffs need a citable basis before the study protocol.
- 本地化临床数据脱敏Runs a local, network-free script that detects and removes identifiers from clinical data. Use when data may contain PHI and must be de-identified before LLM analysis.
- AI对标专家基准研究设计Plans the arms, rubrics, calibration probes and reviewer panel for an AI-vs-expert benchmark. Use before data collection when designing an AI-vs-human rating study.
- 放射影像AI研究设计审查Reviews a study’s analysis unit, cohort logic, leakage risk and reporting-guideline fit. Use when checking a clinical or medical-AI study design before drafting.
- 传染病仓室模型拟合Builds and fits SIR, SEIR and SEIHR compartmental models with Rt estimation. Use when an outbreak or epidemic’s spread needs mathematical modeling in Python.
- ICMJE利益冲突声明填写Clones a pre-filled synthetic ICMJE COI form, replacing only date, name and title. Use when each author on a submission needs a disclosure form filled quickly.
- 机构Word表单内容填充Fills an institutional Word form, such as an IRB protocol, without breaking its layout. Use when drafted content must be rendered into a CJK-aware .docx template.
- 队列数据库选题挖掘Profiles a longitudinal cohort database and ranks research-topic proposals with gap evidence. Use when looking for a question an available cohort or registry can answer.
- 投稿期刊匹配推荐Matches a manuscript abstract against curated journal profiles and ranks candidates. Use when choosing where to submit, or re-targeting after a rejection.
- DOI全文批量获取Batch-downloads open-access PDFs for a list of DOIs via Unpaywall, PMC and OpenAlex. Use when full texts are needed for a screening set such as a meta-analysis.
- 数据集编码本自动生成Profiles every variable in a tabular dataset into a structured, citable codebook. Use when a CSV, Excel or Stata dataset needs a data dictionary before analysis.
- 全球卫生数据获取分析Accesses and analyzes global health data from WHO GHO, IHME GBD and Our World in Data. Use when DALYs, age-standardized rates or choropleth maps are needed.
- 医学AI项目基金申请书撰写Structures a grant proposal’s significance, innovation, approach, aims and milestones. Use when drafting a research funding proposal, including a Korean grant plan.
- 卫生经济学评价建模Builds Markov cohort models with QALYs, PSA and cost-effectiveness planes. Use when a health intervention needs a formal cost-effectiveness evaluation.
- AI写作痕迹识别改写Scans a manuscript for AI writing patterns and rewrites flagged passages carefully. Use when a draft or response letter reads as obviously AI-written.
常见问题
量表信效度验证能做什么?
Runs reliability, factor-structure and invariance checks on a questionnaire or PRO. Use when a scale needs psychometric validation before or after it is used。
如何安装量表信效度验证?
在科搭 Codar 左侧点击「插件」,在「组件」页签搜索「量表信效度验证」并安装;在 Claude Code 或 Codex 中可以发送本页的安装提示词,或下载 Skill 包后解压到 ~/.claude/skills 或 ~/.codex/skills。
量表信效度验证需要哪些权限?
该科研 Skill没有声明额外权限;执行具体操作时仍由 Agent 按自身规则请求确认。
量表信效度验证经过官方审核吗?
是。SpeedSkills 收录的每个版本都会记录内容摘要与权限声明,当前版本已通过安全审核。