科研 Skill

缺失数据插补处理

alterlab-missing-data

Handles missing data by naming the MCAR/MAR/MNAR mechanism, then applying multiple imputation (MICE) or FIML instead of single imputation. Use when a dataset has missing values to treat.

发布者 Codar安全审核通过v0.1.0更新于 2026-11-18

SKILL.md

Missing Data — Name the Mechanism, Impute Multiply, Pool by Rubin's Rules

Skill type: ANALYSIS MODULE. Missing data is not a nuisance to delete or fill with a mean. The discipline: state the missingness mechanism, then use a method whose uncertainty is honest — multiple imputation with Rubin's-rules pooling, or FIML. The dangerous shortcut is single imputation, which treats guessed values as observed and understates standard errors.

Core Mission

STATE THE MECHANISM (MCAR / MAR / MNAR). MULTIPLY IMPUTE AND POOL BY RUBIN'S RULES —
SINGLE IMPUTATION FAKES CERTAINTY IT DOESN'T HAVE.

When to Use This Skill

  • "My dataset has missing values — how should I handle them?"
  • "Should I use multiple imputation? How many imputations?"
  • "How do I pool results across imputed datasets?"
  • "Is mean imputation / listwise deletion okay here?"

Does NOT Trigger

The request is really about…Route toWhy not this skill
Modeling on already-complete dataalterlab-statistical-analysis / alterlab-statsmodelsNo missingness to handle.
FIML inside an SEM / latent-variable modelalterlab-sem-psychometricsThat skill fits the latent model with FIML.
Survey weights / design (a different kind of "adjustment")alterlab-survey-analysisDesign-based inference, not imputation.
Whether the study design is soundalterlab-ssci-design-gateDesign routing.

Hand-off rule. When the request matches a row above, open your answer by naming that skill by its exact id (for example alterlab-statistical-analysis) and saying in one line why it fits better, then stop applying this skill's procedure. Name it even when you also need to ask for missing input. If only part of the request is out of scope, name the skill for that part and do only the in-scope part. A brief general pointer is fine; the named skill does the work.

Running the code. In Claude Code, run analysis code in a throwaway environment instead of installing packages: uv run --python ">=3.11" --with "statsmodels>=0.14" --with pandas python impute.py. This needs no install approval and leaves the user's environment untouched. If uv or the network is unavailable, say so and hand over the script; never report results you did not compute.

Step 1 — state the mechanism (Rubin)

MechanismMeaningConsequence
MCARmissingness independent of everythinglistwise deletion is unbiased (but wasteful)
MARmissingness depends on observed dataMI / FIML are valid (the workhorse assumption)
MNARmissingness depends on the unobserved value itselfneeds a selection / pattern-mixture model + sensitivity analysis

The mechanism is an assumption, largely untestable (MAR vs MNAR especially) — state it and justify it; inspect the missingness pattern (mice::md.pattern) and test MCAR (Little's test) as supporting evidence, not proof.

Step 2 — multiple imputation with Rubin's-rules pooling

Create m completed datasets (each imputes with added noise reflecting uncertainty), analyze each, then pool: the point estimate is the mean across imputations; the SE combines within-imputation and between-imputation variance (Rubin's rules), so it honestly reflects imputation uncertainty.

Python — statsmodels MICE:

import statsmodels.api as sm
from statsmodels.imputation import mice
imp = mice.MICEData(df)                                  # chained-equations imputer over the frame
fit = mice.MICE("y ~ x1 + x2", sm.OLS, imp).fit(n_burnin=10, n_imputations=20)
fit.summary()                                            # pooled estimates + SEs (Rubin's rules)

R — mice (field standard):

library(mice)
md.pattern(data)                                         # inspect the missingness pattern
imp  <- mice(data, m = 20, method = "pmm")               # m completed datasets
fit  <- with(imp, lm(y ~ x1 + x2))
summary(pool(fit))                                       # Rubin's-rules pooling (fmi, lambda, df)

FIML — for likelihood/SEM models under MAR, full-information ML uses all available data without explicit imputation; route to alterlab-sem-psychometrics when the model is latent-variable.

The single-imputation trap (verified caveat)

  • Mean / regression / single imputation treats imputed values as if observed → biased, understated SEs. Not acceptable as a final analysis.
  • scikit-learn IterativeImputer returns a single completed dataset and does not pool by Rubin's rules — the sklearn docs say so explicitly. It is fine for a prediction pipeline, but using it as "multiple imputation" for inference understates uncertainty. If you must, run it repeatedly with sample_posterior=True and different seeds and pool manually — but mice/statsmodels MICE do this correctly out of the box.

How many imputations (m)

Rule of thumb: m at least the percentage of incomplete cases (higher fraction of missing information → larger m); modern guidance often uses m ≈ 20–50. Report m and the fraction of missing information (FMI).

Reporting checklist

MECHANISM:   MCAR / MAR / MNAR — stated and justified; pattern inspected (md.pattern)
METHOD:      MICE (m, imputation model) with Rubin's-rules pooling | FIML | (deletion only if MCAR)
POOLING:     estimates + SEs pooled by Rubin's rules; report FMI
SENSITIVITY: for MNAR, a sensitivity analysis (selection / pattern-mixture)

References

  • references/mechanisms_and_pooling.md — MCAR/MAR/MNAR detail, Rubin's-rules math, FIML vs MI, m guidance, the sklearn caveat.

Part of the AlterLab Academic Skills suite.

示例提示词

我的问卷数据有不少缺失值,想用多重插补法正确处理后再做分析。

所属插件

社会学、政治学与城市研究面向社会科学研究:问卷与访谈设计、质性编码、文本即数据、社会网络、ABM、比较政治与选举数据、RCT 与 DID、城市与区域分析。

安装该插件会一并获得同领域的 58 个 Skill / MCP。

同插件的其他科研 Skill

常见问题

缺失数据插补处理能做什么?

Handles missing data by naming the MCAR/MAR/MNAR mechanism, then applying multiple imputation (MICE) or FIML instead of single imputation. Use when a dataset has missing values to treat。

如何安装缺失数据插补处理?

在科搭 Codar 左侧点击「插件」,在「组件」页签搜索「缺失数据插补处理」并安装;在 Claude Code 或 Codex 中可以发送本页的安装提示词,或下载 Skill 包后解压到 ~/.claude/skills 或 ~/.codex/skills。

缺失数据插补处理需要哪些权限?

该科研 Skill没有声明额外权限;执行具体操作时仍由 Agent 按自身规则请求确认。

缺失数据插补处理经过官方审核吗?

是。SpeedSkills 收录的每个版本都会记录内容摘要与权限声明,当前版本已通过安全审核。