科研 Skill
教育多层线性模型
hierarchical-linear-modeling-edu
Fits multilevel models for students nested in classrooms/schools, with ICC checks and growth models. Use for cluster-randomized trials or nested data.
SKILL.md
Hierarchical Linear Modeling (Multilevel Models) for Education Research
When to Use This Skill
Education data is almost always nested: students within classrooms, within schools, within districts. Treating nested data as if every observation were independent — a plain t-test, OLS regression, or ANOVA on raw student scores — understates standard errors and inflates false-positive rates, because students in the same classroom are more similar to each other than to students elsewhere (the "non-independence" / design-effect problem).
Use this skill when you need to:
- Analyze a cluster-randomized trial (randomization at classroom/school level)
- Partition variance into between-school and within-school components (the intraclass correlation, ICC)
- Estimate school or teacher "value-added" effects while separating them from student-level noise
- Model individual growth trajectories with repeated measures nested within students (a 2-level growth model)
- Decide whether you even need a multilevel model (compute the ICC first)
This skill is the natural complement to learning-experiment: that skill's
A/B-testing example explicitly assumes independent students and flags
clustering as a limitation — this skill is where you go to handle it
properly.
Do NOT use this skill when your data genuinely has no meaningful grouping structure (e.g., a single classroom, or an online platform where "teacher" has no instructional meaning) — a multilevel model with only one or two clusters per group will not converge or will give meaningless variance estimates.
Prerequisites
- Longitudinal or cross-sectional data with an explicit grouping/cluster
identifier (e.g.,
school_id,classroom_id,student_idfor repeated measures) and at least ~20-30 groups at the highest level — fewer makes the group-level variance estimate unstable (a common rule of thumb, not a hard cutoff). - Python 3.10+ with
pandas,numpy,statsmodels(MixedLM) for standard 2-level linear mixed models. For 3+ levels, crossed random effects, or non-Gaussian outcomes (binary, count), prefer R'slme4or Python'sbambi/pymer4, which this skill does not implement itself. - Basic familiarity with fixed vs. random effects and the idea of "partial pooling" / shrinkage, to interpret random-effect estimates correctly.
Limitations
- The examples below use
statsmodels.MixedLM, which supports linear (Gaussian) 2-level models with random intercepts and random slopes. It does not natively fit 3-level models (e.g., students in classrooms in schools) or generalized (logistic/Poisson) multilevel models — for those, use Rlme4/glmeror Pythonbambi(Bayesian multilevel, via PyMC). - Small numbers of level-2 units (<20 schools/classrooms) will often produce a random-effect variance estimate that is unstable or numerically zero even when real clustering exists — do not conclude "no clustering" from a small-sample null result; report the limitation explicitly.
- Centering choices (grand-mean vs. group-mean centering of level-1 predictors) change the substantive interpretation of coefficients; this skill defaults to grand-mean centering and flags where that matters.
- This skill does not cover multilevel structural equation modeling or
multilevel CFA/measurement invariance — pair with
scale-validation-cfaconceptually, but note neither skill implements that combination.
Environment Setup
pip install numpy pandas scipy statsmodels matplotlib
# For 3+ level or generalized multilevel models (recommended for real analyses):
# R: install.packages(c("lme4", "lmerTest", "performance"))
# Python Bayesian alternative: pip install bambi
Step 1 — Should You Even Use a Multilevel Model? Compute the ICC
import numpy as np
import pandas as pd
import statsmodels.formula.api as smf
def intraclass_correlation(df: pd.DataFrame, outcome: str, group_col: str) -> dict:
"""
Fit a null (intercept-only) random-intercept model and compute the ICC:
the proportion of total variance that lies between groups.
ICC = tau^2 / (tau^2 + sigma^2)
Rules of thumb: ICC > 0.05-0.10 in education data is common and usually
large enough that ignoring clustering meaningfully biases standard
errors; always report it rather than assuming it away.
Args:
df: long-format data, one row per student (or per observation).
outcome: name of the outcome column.
group_col: name of the clustering column (e.g., 'classroom_id').
Returns:
dict with icc, between_group_var (tau2), within_group_var (sigma2),
design_effect (for a given average cluster size n_bar).
"""
model = smf.mixedlm(f"{outcome} ~ 1", data=df, groups=df[group_col])
fit = model.fit(reml=True)
tau2 = float(fit.cov_re.iloc[0, 0])
sigma2 = float(fit.scale)
icc = tau2 / (tau2 + sigma2)
cluster_sizes = df.groupby(group_col).size()
n_bar = cluster_sizes.mean()
design_effect = 1 + (n_bar - 1) * icc
return {
"icc": round(icc, 4),
"between_group_var": round(tau2, 4),
"within_group_var": round(sigma2, 4),
"avg_cluster_size": round(n_bar, 1),
"design_effect": round(design_effect, 3),
"effective_n_if_naive": round(len(df) / design_effect, 1),
}
If the design effect is, say, 2.5, a naive analysis that assumes len(df)
independent observations is overstating the effective sample size by a
factor of 2.5 — a very common and serious error in school-based research.
Step 2 — Random-Intercept Model with a Treatment Effect
def fit_random_intercept_model(
df: pd.DataFrame,
outcome: str,
fixed_effects: list,
group_col: str,
) -> dict:
"""
Fit a 2-level random-intercept model: fixed effects at level 1 (student)
plus a random intercept at level 2 (e.g., classroom or school).
Args:
df: long-format data.
outcome: outcome column name.
fixed_effects: list of predictor column names (e.g., ['treat', 'pre_test']).
group_col: clustering column.
Returns:
dict with model summary table, fixed effect estimates + cluster-aware
SEs, and the random-intercept variance.
"""
formula = f"{outcome} ~ " + " + ".join(fixed_effects)
model = smf.mixedlm(formula, data=df, groups=df[group_col])
fit = model.fit(reml=True)
return {
"params": fit.params,
"bse": fit.bse,
"pvalues": fit.pvalues,
"random_intercept_var": float(fit.cov_re.iloc[0, 0]),
"residual_var": float(fit.scale),
"summary": fit.summary(),
}
Step 3 — Random Slopes (Does the Treatment Effect Vary by Cluster?)
def fit_random_slope_model(
df: pd.DataFrame,
outcome: str,
fixed_effects: list,
random_slope_var: str,
group_col: str,
) -> dict:
"""
Fit a model where the effect of `random_slope_var` (e.g., 'treat') is
allowed to vary by cluster, in addition to a random intercept.
Use this when you suspect treatment effects differ meaningfully across
classrooms/schools/teachers (effect heterogeneity), not just baseline
performance.
Args:
df: long-format data.
outcome: outcome column.
fixed_effects: predictor column names for the fixed-effects part.
random_slope_var: predictor whose slope is allowed to vary by group.
group_col: clustering column.
Returns:
dict with fixed effects, random effects covariance structure,
and a likelihood-ratio-test helper against the random-intercept-only model.
"""
formula = f"{outcome} ~ " + " + ".join(fixed_effects)
model = smf.mixedlm(
formula, data=df, groups=df[group_col],
re_formula=f"~{random_slope_var}",
)
fit = model.fit(reml=False) # ML (not REML) needed for valid LRT comparisons
return {
"params": fit.params,
"bse": fit.bse,
"cov_re": fit.cov_re,
"llf": fit.llf,
"summary": fit.summary(),
}
def likelihood_ratio_test(llf_simple: float, llf_complex: float, df_diff: int) -> dict:
"""
Likelihood ratio test comparing nested models (both fit with ML, not REML).
Args:
llf_simple: log-likelihood of the simpler (nested) model.
llf_complex: log-likelihood of the more complex model.
df_diff: difference in number of parameters.
Returns:
dict with LR statistic, df, and p-value.
"""
from scipy.stats import chi2
lr_stat = 2 * (llf_complex - llf_simple)
p_value = 1 - chi2.cdf(lr_stat, df=df_diff)
return {"LR_stat": round(lr_stat, 3), "df": df_diff, "p_value": round(p_value, 4)}
Step 4 — Growth Modeling (Repeated Measures Nested in Students)
def fit_growth_model(df: pd.DataFrame, outcome: str, time_col: str, student_col: str) -> dict:
"""
2-level growth model: repeated measurement occasions (level 1) nested
within students (level 2). Random intercept and random slope for time
capture individual differences in starting level and growth rate.
Args:
df: long-format data with one row per student per occasion.
outcome: outcome column (e.g., reading score).
time_col: numeric time variable (e.g., 0, 1, 2 for 3 waves).
student_col: student identifier.
Returns:
dict with average intercept/slope (fixed effects) and the
between-student variance in intercepts and slopes.
"""
formula = f"{outcome} ~ {time_col}"
model = smf.mixedlm(
formula, data=df, groups=df[student_col], re_formula=f"~{time_col}",
)
fit = model.fit(reml=True)
return {
"avg_intercept": float(fit.params["Intercept"]),
"avg_slope": float(fit.params[time_col]),
"intercept_variance": float(fit.cov_re.iloc[0, 0]),
"slope_variance": float(fit.cov_re.iloc[1, 1]) if fit.cov_re.shape[0] > 1 else None,
"summary": fit.summary(),
}
Example — Cluster-Randomized Trial Analyzed Correctly
import numpy as np
import pandas as pd
np.random.seed(42)
n_classrooms = 40
students_per_classroom = 20
# Randomize at the CLASSROOM level (not the student level)
classroom_ids = np.arange(n_classrooms)
classroom_treat = np.random.binomial(1, 0.5, n_classrooms)
# True classroom-level random effect (some classrooms just do better)
classroom_effect = np.random.normal(0, 4, n_classrooms)
rows = []
for c in classroom_ids:
pre = np.random.normal(55, 12, students_per_classroom)
treatment_effect = 6 if classroom_treat[c] else 0
post = (pre * 0.75 + treatment_effect + classroom_effect[c]
+ np.random.normal(0, 9, students_per_classroom))
for p, po in zip(pre, post):
rows.append({
"classroom_id": c, "treat": classroom_treat[c],
"pre_test": p, "post_test": po,
})
df = pd.DataFrame(rows)
# Step 1: check the ICC on the outcome
icc_info = intraclass_correlation(df, outcome="post_test", group_col="classroom_id")
print("ICC / design effect check:", icc_info)
# Step 2: naive (WRONG) analysis ignoring clustering
import statsmodels.api as sm
X_naive = sm.add_constant(df[["treat", "pre_test"]])
naive_fit = sm.OLS(df["post_test"], X_naive).fit()
print(f"\nNaive OLS treatment SE (ignores clustering): {naive_fit.bse['treat']:.3f}")
# Step 3: correct multilevel analysis
mlm_result = fit_random_intercept_model(
df, outcome="post_test", fixed_effects=["treat", "pre_test"], group_col="classroom_id",
)
print(f"\nMultilevel model treatment effect: {mlm_result['params']['treat']:.3f}")
print(f"Multilevel model treatment SE (accounts for clustering): {mlm_result['bse']['treat']:.3f}")
print(f"Random intercept variance (between-classroom): {mlm_result['random_intercept_var']:.3f}")
# The multilevel SE for 'treat' should be noticeably larger than the naive
# OLS SE whenever ICC > 0 and randomization was at the classroom level —
# that gap is exactly the bias the naive analysis would have hidden.
Troubleshooting
| Problem | Likely Cause | Fix |
|---|---|---|
MixedLM fails to converge | Too few groups, near-zero random-effect variance, or scaling issues | Standardize predictors; try reml=False; reduce random-effects complexity |
| Random-intercept variance estimated as ~0 | Genuinely low clustering, or too few groups to detect it | Report honestly; do not force a random effect that the data can't support |
| Naive and multilevel SEs look identical | ICC is near zero for this outcome, OR groups were not actually the randomization unit | Verify ICC; verify the real unit of randomization |
| Random slope model won't converge | Too few observations per cluster to estimate a slope per cluster | Drop to random intercept only, or increase per-cluster sample size |
| P-values look too good from REML LRT | REML is not valid for comparing fixed-effects-differing models | Refit both models with reml=False before an LRT comparing fixed effects |
Quality Checks Before Reporting a Multilevel Analysis
- Always report the ICC / design effect for the outcome, even if you end up deciding to proceed with simpler methods.
- Fit the null (intercept-only) model first to establish the baseline variance decomposition before adding predictors.
- Use REML for final variance-component estimates, but ML when comparing nested models that differ in fixed effects via likelihood ratio tests.
- Report at least 20+ level-2 units; if fewer, say so explicitly as a limitation rather than presenting variance estimates as if precise.
- Check residual diagnostics at both levels (level-1 residuals and the predicted random effects/BLUPs) for major departures from normality before trusting p-values in borderline cases.
External Resources
- Raudenbush, S. W., & Bryk, A. S. (2002). Hierarchical Linear Models (2nd ed.). Sage.
- Snijders, T. A. B., & Bosker, R. J. (2012). Multilevel Analysis (2nd ed.). Sage.
statsmodelsMixedLM docs: https://www.statsmodels.org/stable/mixed_linear.html- R
lme4package (reference implementation for generalized/complex models): https://cran.r-project.org/package=lme4 - What Works Clearinghouse standards for cluster-randomized trials: https://ies.ed.gov/ncee/wwc/
示例提示词
我的实验按班级整群随机分组,想用多层线性模型正确估计处理效应。
所属插件
安装该插件会一并获得同领域的 7 个 Skill / MCP。
同插件的其他科研 Skill
- 教育数据挖掘分析Mines LMS logs with Bayesian knowledge tracing, Cox dropout survival models, IRT, and sequence mining. Use to predict at-risk students or trace skill mastery.
- 教育研究元分析Pools effect sizes across studies with random-effects models and heterogeneity stats. Use for a systematic review of intervention effects across studies.
- 教育测量项目反应理论Calibrates 2PL/3PL IRT models and plots item/test information functions and DIF. Use when calibrating test items or auditing fairness across groups.
- 教育心理学实验设计分析Designs and analyzes education experiments: RCTs, IRT scoring, growth/forgetting curves, A/B tests. Use when planning or analyzing an intervention study.
- PISA国际测评数据分析Analyzes PISA/TIMSS data with plausible-value averaging, BRR variance, and cross-national SES gradients. Use to compare country performance from assessment data.
- 量表验证与因子分析Validates survey scales with EFA/CFA, alpha/omega reliability, and measurement invariance. Use when piloting a motivation or attitude instrument.
常见问题
教育多层线性模型能做什么?
Fits multilevel models for students nested in classrooms/schools, with ICC checks and growth models. Use for cluster-randomized trials or nested data。
如何安装教育多层线性模型?
在科搭 Codar 左侧点击「插件」,在「组件」页签搜索「教育多层线性模型」并安装;在 Claude Code 或 Codex 中可以发送本页的安装提示词,或下载 Skill 包后解压到 ~/.claude/skills 或 ~/.codex/skills。
教育多层线性模型需要哪些权限?
该科研 Skill没有声明额外权限;执行具体操作时仍由 Agent 按自身规则请求确认。
教育多层线性模型经过官方审核吗?
是。SpeedSkills 收录的每个版本都会记录内容摘要与权限声明,当前版本已通过安全审核。