科研 Skill
社科推论结论审核
alterlab-ssci-inference-gate
Audits inferential claims against the design before publication: unwarranted causal language, p-value/CI misreadings, and multiple-comparison issues. Use when checking a results section.
SKILL.md
Inference Gate — The Claim May Not Exceed the Design
Skill type: DISCIPLINE-ENFORCING. This is the terminal gate of the social-science methods spine. It reads the accumulated Design Passport and audits every inferential sentence against what actually identifies it — the design, the sample, and the uncertainty. It does not compute; it refuses conclusions the study cannot support.
The Core Rule
A CLAIM MAY NOT EXCEED ITS DESIGN, ITS SAMPLE, OR ITS UNCERTAINTY.
AUDIT EACH SENTENCE AGAINST WHAT ACTUALLY IDENTIFIES IT.
Three ceilings bound every claim. Design: causal language is licensed only if the design's
identifying assumption (pinned by alterlab-ssci-design-gate) is named and defended — otherwise
the claim is associational. Sample: a population generalization is licensed only by a
probability sample of a frame that covers the target population (alterlab-ssci-sampling-gate).
Uncertainty: a result is reported with an effect size and an interval, interpreted correctly —
not as a bare significant/not-significant verdict. A sentence that breaks any ceiling is rewritten
down to what the study supports.
When to Use This Skill
- "Help me write / check the results or discussion section."
- "My p-value is 0.03 — what can I conclude?" / "p was 0.20, so there's no effect, right?"
- "Does this finding mean X causes Y?"
- "Can I say this holds for [the broader population]?"
- "I ran 30 comparisons and three were significant — what can I report?"
Does NOT Trigger
| The request is really about… | Route to | Why not this skill |
|---|---|---|
| Which statistical test to run | alterlab-test-selection-guard | Test choice, upstream of interpretation. |
| Executing the analysis / computing the estimate | alterlab-statistical-analysis | Computation, not claim audit. |
| Choosing the design & identifying assumption | alterlab-ssci-design-gate | Design routing happens first. |
| Reliability/validity of the measure | alterlab-ssci-measurement-gate | Measurement quality, a different ceiling. |
| Prose polish of an already-sound claim | alterlab-paper-writer | Writing, once the claim is audited clean. |
Hand-off rule. When the request matches a row above, open your answer by naming that skill by its exact id (for example alterlab-test-selection-guard) and saying in one line why it fits better, then stop applying this skill's procedure. Name it even when you also need to ask for missing input. If only part of the request is out of scope, name the skill for that part and do only the in-scope part. A brief general pointer is fine; the named skill does the work.
The three ceilings
1. Design ceiling — causal language
Cross-check the claim's verb against design_type + identifying_assumption in the Passport:
- Causal verbs (causes, increases, reduces, improves, leads to, the effect of) require a design that identifies a causal effect and a defended assumption. Experiment → yes. QED → yes only if its assumption (parallel trends / exclusion / continuity / no time-varying confounders) is defended. Observational without a defended assumption → downgrade to associational (is associated with, predicts, correlates with).
- Never let a significant coefficient in an observational regression become "X causes Y."
2. Sample ceiling — generalization
- Statistical generalization to a population requires a probability sample of a covering frame
(
generalization: statisticalin the Passport). Otherwise scope the claim to the sample ("among the surveyed students…") and offer only analytical generalization. - Watch universal quantifiers ("adults", "people generally") attached to a convenience sample.
3. Uncertainty ceiling — p-values, CIs, effect sizes
| Misreading | Correction |
|---|---|
| "p = 0.03 means a 3% chance the null is true." | p is P(data this extreme | null), not P(null | data). |
| "p > 0.05, so there is no effect." | Absence of evidence ≠ evidence of absence. Report the CI; a wide CI around zero is inconclusive, not a null proof. |
| "The 95% CI means a 95% probability the true value is in it." | The parameter is fixed; 95% is the long-run coverage of the procedure. |
| "It's statistically significant, so it matters." | Significance ≠ importance. Report the effect size and its interval; judge practical significance against it. |
| "Three of my 30 tests were significant." | Uncorrected multiplicity. Pre-specify, or correct (FDR / Bonferroni); disclose the family. |
| "I found the hypothesis the data supported." | HARKing / optional stopping. Separate pre-registered confirmatory from exploratory claims. |
A stdlib linter that flags causal verbs, over-broad generalization, and p-value misuse against the
Passport facts: scripts/claim_audit.py. Fuller treatment (ASA statement on p-values, New
Statistics, multiplicity): references/inference_audit.md.
Reading the Design Passport
The gate consumes what upstream gates wrote and audits against it:
# from design-gate
design_type: observational
identifying_assumption: "conditional ignorability — NOT defended (no plausible ignorability argument)"
claim_type: associational
# from sampling-gate
sampling_method: convenience
generalization: analytical
Given this Passport, "remote work increases satisfaction among adults" fails two ceilings and is rewritten to "remote work is associated with higher satisfaction among the surveyed employees."
Self-Check Before Publishing
- Does every causal verb trace to a design + a defended identifying assumption?
- Is each generalization scoped to what the sample/frame supports (statistical vs analytical)?
- Are results reported as effect size + interval, with p-values interpreted correctly?
- Is a non-significant result framed as inconclusive (CI shown), not as proof of no effect?
- Are multiplicity and any exploratory/optional-stopping caveats disclosed?
References
references/inference_audit.md— p-value/CI interpretation, multiplicity correction, the three-ceiling audit worked through.scripts/claim_audit.py— stdlib linter flagging causal-verb, generalization, and p-value overreach against Passport facts.
Part of the AlterLab Academic Skills suite.
示例提示词
帮我检查这篇论文的因果表述和p值解读是否超出研究设计的支持范围。
所属插件
安装该插件会一并获得同领域的 58 个 Skill / MCP。
同插件的其他科研 Skill
- Mesa社会主体建模Builds Mesa agent-based models of opinion dynamics, segregation, social contagion, and network diffusion. Use when simulating emergent social behavior from agent rules.
- Mesa3主体建模引擎Builds agent-based models with the Mesa 3 AgentSet API, cell spaces, and batch_run sweeps for segregation, diffusion, and cooperation. Use for a Mesa 3.x ABM build or debug.
- 观察数据因果推断Estimates causal effects with DiD, IV, RDD, panel fixed effects, and propensity scores via statsmodels, linearmodels, DoWhy, and EconML. Use for observational or quasi-experimental causal estimation.
- 效应量元分析Pools effect sizes across studies with fixed/random-effects models, heterogeneity stats, forest/funnel plots, and publication-bias tests. Use for a systematic review’s quantitative synthesis.
- 缺失数据插补处理Handles missing data by naming the MCAR/MAR/MNAR mechanism, then applying multiple imputation (MICE) or FIML instead of single imputation. Use when a dataset has missing values to treat.
- 多层线性混合模型Fits mixed-effects/multilevel models (random slopes, ICC, GLMMs) for nested or longitudinal data via statsmodels, bambi, or R lme4/brms. Use for grouped data with within- vs between-cluster questions.
- 定性比较分析QCARuns crisp-, multi-value, and fuzzy-set Qualitative Comparative Analysis via R’s QCA package. Use for configurational or necessary/sufficient-condition research.
- 质性编码与主题分析Builds codebooks and computes intercoder reliability (Krippendorff’s alpha, kappa) for thematic or content coding, including human-vs-LLM double coding. Use when coding text or reporting reliability.
- 社会网络结构分析Computes network centrality, community detection, and ERGM inferential models matched to the research question via networkx/igraph. Use for social-network analysis beyond basic visualization.
- 社科研究设计把关Routes a study to its research design (experiment, quasi-experiment, observational, qualitative) and pins the identifying assumption first. Use when choosing a study design.
- 量表测量质量把关Checks that a scale’s reliability, validity, and measurement invariance are established before trusting or comparing it across groups. Use when adopting or reporting a survey instrument.
- 社科研究流程总控Coordinates a study through design, measurement, sampling, analysis, and inference gates with a shared Design Passport. Use for an end-to-end methodology review of a whole study.
- 质性研究可信度把关Gates qualitative trustworthiness before write-up: positionality, credibility, transferability, dependability, confirmability per Lincoln and Guba. Use before writing ethnographic findings.
- 抽样方法与样本量把关Checks that the sampling frame, method, and size match the inference paradigm before data collection. Use when planning recruitment or judging sample generalization.
- 复杂抽样调查分析Analyzes complex-sample survey data with design-based inference: weights, strata, PSUs, and calibration via Python svy or R survey/srvyr. Use for weighted surveys like GSS, ANES, or DHS.
- 文本数据定量分析Analyzes text as social-science data with topic modeling, embeddings, and validated dictionary or classification methods. Use for computational text analysis beyond a keyword count.
- 政策议题框架分析Runs anthropological framing and metaphor analysis on contested-policy documents, producing actor-attributed CSVs for NVivo import. Use for policy-framing or metaphor-analysis coding.
- 跨国比较政治分析Merges QoG, Polity5, and Freedom House data and runs cross-sectional OLS with fixed effects and regime classification. Use for comparative political-institution research.
- 计算社会学方法Covers social-media data collection, bot detection, network homophily, echo-chamber analysis, and conjoint survey experiments. Use for large-scale digital-trace sociology research.
- 冲突事件数据分析Downloads ACLED/UCDP conflict data and runs spatio-temporal clustering, conflict-onset logit panels, and fatality estimation. Use for quantitative armed-conflict research.
- 联合分析调查实验Designs conjoint survey experiments and estimates AMCEs via OLS, marginal means, and respondent-level heterogeneity. Use when designing or analyzing a choice-based conjoint experiment.
- 统计方法教学规范Defines the baseline statistical teaching and reproducibility standards shared across this analysis skill family. Use as the methodology reference other skills in the family build on.
- 人口统计方法分析Computes life tables, Leslie-matrix population projections, Lee-Carter mortality forecasts, and Kitagawa decomposition. Use for formal demographic analysis of mortality or population structure.
- 描述性统计分析Produces summary statistics, distributions, correlations, and cross-tabs for exploratory analysis. Use when summarizing a dataset before modeling or reporting.
常见问题
社科推论结论审核能做什么?
Audits inferential claims against the design before publication: unwarranted causal language, p-value/CI misreadings, and multiple-comparison issues. Use when checking a results section。
如何安装社科推论结论审核?
在科搭 Codar 左侧点击「插件」,在「组件」页签搜索「社科推论结论审核」并安装;在 Claude Code 或 Codex 中可以发送本页的安装提示词,或下载 Skill 包后解压到 ~/.claude/skills 或 ~/.codex/skills。
社科推论结论审核需要哪些权限?
该科研 Skill没有声明额外权限;执行具体操作时仍由 Agent 按自身规则请求确认。
社科推论结论审核经过官方审核吗?
是。SpeedSkills 收录的每个版本都会记录内容摘要与权限声明,当前版本已通过安全审核。