科研 Skill

社科推论结论审核

alterlab-ssci-inference-gate

Audits inferential claims against the design before publication: unwarranted causal language, p-value/CI misreadings, and multiple-comparison issues. Use when checking a results section.

发布者 Codar安全审核通过v0.1.0更新于 2026-11-18

SKILL.md

Inference Gate — The Claim May Not Exceed the Design

Skill type: DISCIPLINE-ENFORCING. This is the terminal gate of the social-science methods spine. It reads the accumulated Design Passport and audits every inferential sentence against what actually identifies it — the design, the sample, and the uncertainty. It does not compute; it refuses conclusions the study cannot support.

The Core Rule

A CLAIM MAY NOT EXCEED ITS DESIGN, ITS SAMPLE, OR ITS UNCERTAINTY.
AUDIT EACH SENTENCE AGAINST WHAT ACTUALLY IDENTIFIES IT.

Three ceilings bound every claim. Design: causal language is licensed only if the design's identifying assumption (pinned by alterlab-ssci-design-gate) is named and defended — otherwise the claim is associational. Sample: a population generalization is licensed only by a probability sample of a frame that covers the target population (alterlab-ssci-sampling-gate). Uncertainty: a result is reported with an effect size and an interval, interpreted correctly — not as a bare significant/not-significant verdict. A sentence that breaks any ceiling is rewritten down to what the study supports.

When to Use This Skill

  • "Help me write / check the results or discussion section."
  • "My p-value is 0.03 — what can I conclude?" / "p was 0.20, so there's no effect, right?"
  • "Does this finding mean X causes Y?"
  • "Can I say this holds for [the broader population]?"
  • "I ran 30 comparisons and three were significant — what can I report?"

Does NOT Trigger

The request is really about…Route toWhy not this skill
Which statistical test to runalterlab-test-selection-guardTest choice, upstream of interpretation.
Executing the analysis / computing the estimatealterlab-statistical-analysisComputation, not claim audit.
Choosing the design & identifying assumptionalterlab-ssci-design-gateDesign routing happens first.
Reliability/validity of the measurealterlab-ssci-measurement-gateMeasurement quality, a different ceiling.
Prose polish of an already-sound claimalterlab-paper-writerWriting, once the claim is audited clean.

Hand-off rule. When the request matches a row above, open your answer by naming that skill by its exact id (for example alterlab-test-selection-guard) and saying in one line why it fits better, then stop applying this skill's procedure. Name it even when you also need to ask for missing input. If only part of the request is out of scope, name the skill for that part and do only the in-scope part. A brief general pointer is fine; the named skill does the work.

The three ceilings

1. Design ceiling — causal language

Cross-check the claim's verb against design_type + identifying_assumption in the Passport:

  • Causal verbs (causes, increases, reduces, improves, leads to, the effect of) require a design that identifies a causal effect and a defended assumption. Experiment → yes. QED → yes only if its assumption (parallel trends / exclusion / continuity / no time-varying confounders) is defended. Observational without a defended assumption → downgrade to associational (is associated with, predicts, correlates with).
  • Never let a significant coefficient in an observational regression become "X causes Y."

2. Sample ceiling — generalization

  • Statistical generalization to a population requires a probability sample of a covering frame (generalization: statistical in the Passport). Otherwise scope the claim to the sample ("among the surveyed students…") and offer only analytical generalization.
  • Watch universal quantifiers ("adults", "people generally") attached to a convenience sample.

3. Uncertainty ceiling — p-values, CIs, effect sizes

MisreadingCorrection
"p = 0.03 means a 3% chance the null is true."p is P(data this extreme | null), not P(null | data).
"p > 0.05, so there is no effect."Absence of evidence ≠ evidence of absence. Report the CI; a wide CI around zero is inconclusive, not a null proof.
"The 95% CI means a 95% probability the true value is in it."The parameter is fixed; 95% is the long-run coverage of the procedure.
"It's statistically significant, so it matters."Significance ≠ importance. Report the effect size and its interval; judge practical significance against it.
"Three of my 30 tests were significant."Uncorrected multiplicity. Pre-specify, or correct (FDR / Bonferroni); disclose the family.
"I found the hypothesis the data supported."HARKing / optional stopping. Separate pre-registered confirmatory from exploratory claims.

A stdlib linter that flags causal verbs, over-broad generalization, and p-value misuse against the Passport facts: scripts/claim_audit.py. Fuller treatment (ASA statement on p-values, New Statistics, multiplicity): references/inference_audit.md.

Reading the Design Passport

The gate consumes what upstream gates wrote and audits against it:

# from design-gate
design_type: observational
identifying_assumption: "conditional ignorability — NOT defended (no plausible ignorability argument)"
claim_type: associational
# from sampling-gate
sampling_method: convenience
generalization: analytical

Given this Passport, "remote work increases satisfaction among adults" fails two ceilings and is rewritten to "remote work is associated with higher satisfaction among the surveyed employees."

Self-Check Before Publishing

  • Does every causal verb trace to a design + a defended identifying assumption?
  • Is each generalization scoped to what the sample/frame supports (statistical vs analytical)?
  • Are results reported as effect size + interval, with p-values interpreted correctly?
  • Is a non-significant result framed as inconclusive (CI shown), not as proof of no effect?
  • Are multiplicity and any exploratory/optional-stopping caveats disclosed?

References

  • references/inference_audit.md — p-value/CI interpretation, multiplicity correction, the three-ceiling audit worked through.
  • scripts/claim_audit.py — stdlib linter flagging causal-verb, generalization, and p-value overreach against Passport facts.

Part of the AlterLab Academic Skills suite.

示例提示词

帮我检查这篇论文的因果表述和p值解读是否超出研究设计的支持范围。

所属插件

社会学、政治学与城市研究面向社会科学研究:问卷与访谈设计、质性编码、文本即数据、社会网络、ABM、比较政治与选举数据、RCT 与 DID、城市与区域分析。

安装该插件会一并获得同领域的 58 个 Skill / MCP。

同插件的其他科研 Skill

常见问题

社科推论结论审核能做什么?

Audits inferential claims against the design before publication: unwarranted causal language, p-value/CI misreadings, and multiple-comparison issues. Use when checking a results section。

如何安装社科推论结论审核?

在科搭 Codar 左侧点击「插件」,在「组件」页签搜索「社科推论结论审核」并安装;在 Claude Code 或 Codex 中可以发送本页的安装提示词,或下载 Skill 包后解压到 ~/.claude/skills 或 ~/.codex/skills。

社科推论结论审核需要哪些权限?

该科研 Skill没有声明额外权限;执行具体操作时仍由 Agent 按自身规则请求确认。

社科推论结论审核经过官方审核吗?

是。SpeedSkills 收录的每个版本都会记录内容摘要与权限声明,当前版本已通过安全审核。