科研 Skill
教育研究元分析
edu-meta-analysis
Pools effect sizes across studies with random-effects models and heterogeneity stats. Use for a systematic review of intervention effects across studies.
SKILL.md
Systematic Review and Meta-Analysis for Education Research
When to Use This Skill
Use this skill when you need to synthesize findings across multiple
education studies rather than run a single new study — for example,
"what is the average effect of feedback interventions on student
achievement across the published literature?" or "is the effect of
class-size reduction consistent across countries?" This complements the
single-study skills in this plugin (learning-experiment,
hierarchical-linear-modeling-edu) by operating one level up, on a
collected set of effect sizes rather than raw student data.
Typical tasks:
- Search and screen studies following a PRISMA-style workflow
- Extract or compute a common effect size (Cohen's d / Hedges' g, or a correlation) from each study's reported statistics
- Pool effect sizes with a random-effects (or fixed-effect) model
- Quantify and visualize heterogeneity across studies (I², τ², Q-test)
- Check for publication bias (funnel plot, Egger's test, trim-and-fill)
- Explore moderators (e.g., grade level, intervention duration) via meta-regression or subgroup analysis
Do NOT use this skill to combine studies that measure conceptually different constructs just because they both report a p-value — meta- analysis assumes the pooled studies estimate a common (or at least related, explainable) underlying effect. Combining apples and oranges produces a confident-looking but meaningless pooled number.
Prerequisites
- A set of studies identified through a documented, reproducible search (database, search string, date range, inclusion/exclusion criteria) — this skill does not perform literature search itself; pair it with database access (ERIC, PsycINFO, Web of Science, Google Scholar) or a reference manager export.
- For each included study: enough reported statistics to compute or extract an effect size and its variance (e.g., group means + SDs + n, or a reported t/F statistic + n, or a correlation + n).
- Python 3.10+ with
numpy,pandas,scipy,matplotlib; thepymareor Rmetaforpackage is recommended for anything beyond a single simple random-effects model (subgroup meta-regression, multivariate/multilevel meta-analysis, cluster-robust variance). - Familiarity with the direction/sign convention for effect sizes across your included studies — a frequent, serious error is mixing studies where "positive" means opposite things (e.g., effect on dropout vs. effect on retention) without re-signing them consistently first.
Limitations
- The pooling functions below implement a standard univariate
random-effects model (DerSimonian-Laird) for independent effect sizes.
They do not handle studies that contribute multiple, dependent effect
sizes (e.g., several outcomes from the same sample) — that requires
robust variance estimation (RVE) or multilevel meta-analysis, which is
not implemented here; use R
metafor/robumetaor Pythonpymarefor that case, and do not naively pool dependent effects as if independent. - Fixed-effect vs. random-effects is a substantive choice, not just a technical default: fixed-effect assumes one true effect underlies every study, which is rarely plausible in education research with varying populations, interventions, and settings. This skill defaults to random-effects; switch only with a clear justification.
- Publication bias diagnostics (funnel plot asymmetry, Egger's test) are low-powered with fewer than ~10 studies and are suggestive, not definitive — do not present them as proof of (or exoneration from) publication bias with a small set of studies.
- This skill does not conduct systematic search or screening itself, and does not substitute for a PRISMA flow diagram or a pre-registered protocol (e.g., in PROSPERO) when one is required by your target venue.
Environment Setup
pip install numpy pandas scipy matplotlib
# Recommended for subgroup/meta-regression and robust variance estimation:
pip install pymare
# R alternative (field standard for complex designs):
# install.packages(c("metafor", "robumeta"))
Step 1 — Compute Effect Sizes from Common Reported Statistics
import numpy as np
import pandas as pd
def smd_from_means(m1, sd1, n1, m2, sd2, n2) -> dict:
"""
Standardized mean difference (Cohen's d) with small-sample correction
(Hedges' g) and its variance, from two group means/SDs/sizes.
Args:
m1, sd1, n1: treatment group mean, SD, sample size.
m2, sd2, n2: control group mean, SD, sample size.
Returns:
dict with d, g, se, variance, n_total.
"""
pooled_sd = np.sqrt(((n1 - 1) * sd1**2 + (n2 - 1) * sd2**2) / (n1 + n2 - 2))
d = (m1 - m2) / pooled_sd
# Hedges' small-sample correction factor
df = n1 + n2 - 2
j = 1 - 3 / (4 * df - 1)
g = d * j
var_d = (n1 + n2) / (n1 * n2) + d**2 / (2 * (n1 + n2))
var_g = (j**2) * var_d
return {"d": d, "g": g, "se": np.sqrt(var_g), "variance": var_g, "n_total": n1 + n2}
def smd_from_t_stat(t_stat, n1, n2) -> dict:
"""Convert an independent-samples t-statistic to Hedges' g."""
d = t_stat * np.sqrt(1 / n1 + 1 / n2)
df = n1 + n2 - 2
j = 1 - 3 / (4 * df - 1)
g = d * j
var_g = (j**2) * ((n1 + n2) / (n1 * n2) + d**2 / (2 * (n1 + n2)))
return {"d": d, "g": g, "se": np.sqrt(var_g), "variance": var_g, "n_total": n1 + n2}
def r_to_z(r: float, n: int) -> dict:
"""Fisher's z transformation of a correlation, with its variance (1/(n-3))."""
z = 0.5 * np.log((1 + r) / (1 - r))
var_z = 1 / (n - 3)
return {"z": z, "se": np.sqrt(var_z), "variance": var_z, "n": n}
def z_to_r(z: float) -> float:
"""Back-transform Fisher's z to a correlation for reporting."""
return (np.exp(2 * z) - 1) / (np.exp(2 * z) + 1)
Step 2 — Random-Effects Pooling (DerSimonian-Laird)
def pool_random_effects(effect_sizes: np.ndarray, variances: np.ndarray) -> dict:
"""
DerSimonian-Laird random-effects meta-analysis.
Args:
effect_sizes: array of study effect sizes (e.g., Hedges' g per study).
variances: array of each effect size's sampling variance.
Returns:
dict with pooled_effect, se, ci_low, ci_high, tau2, Q, I2, p_heterogeneity.
"""
k = len(effect_sizes)
weights_fixed = 1 / variances
# Fixed-effect pooled estimate (needed for Q statistic)
pooled_fixed = np.sum(weights_fixed * effect_sizes) / np.sum(weights_fixed)
Q = np.sum(weights_fixed * (effect_sizes - pooled_fixed) ** 2)
df = k - 1
from scipy.stats import chi2
p_heterogeneity = 1 - chi2.cdf(Q, df) if df > 0 else np.nan
# Tau^2 (between-study variance) via DerSimonian-Laird method of moments
c = np.sum(weights_fixed) - np.sum(weights_fixed**2) / np.sum(weights_fixed)
tau2 = max(0.0, (Q - df) / c) if c > 0 else 0.0
# Random-effects weights and pooled estimate
weights_re = 1 / (variances + tau2)
pooled_re = np.sum(weights_re * effect_sizes) / np.sum(weights_re)
se_re = np.sqrt(1 / np.sum(weights_re))
i2 = max(0.0, (Q - df) / Q) * 100 if Q > 0 else 0.0
return {
"k_studies": k,
"pooled_effect": round(pooled_re, 4),
"se": round(se_re, 4),
"ci_low": round(pooled_re - 1.96 * se_re, 4),
"ci_high": round(pooled_re + 1.96 * se_re, 4),
"tau2": round(tau2, 4),
"Q": round(Q, 3),
"df": df,
"I2_percent": round(i2, 1),
"p_heterogeneity": round(p_heterogeneity, 4) if not np.isnan(p_heterogeneity) else None,
}
I² interpretation (Higgins et al., 2003): ~25% low, ~50% moderate, ~75% high heterogeneity. High I² means a single pooled number may be misleading — look for moderators (Step 4) rather than reporting one average effect.
Step 3 — Forest Plot and Funnel Plot
import matplotlib.pyplot as plt
def forest_plot(study_labels, effects, variances, pooled: dict, output_path: str = "forest_plot.png"):
"""
Forest plot of individual study effects and the pooled random-effects estimate.
"""
ses = np.sqrt(variances)
y = np.arange(len(effects))[::-1]
fig, ax = plt.subplots(figsize=(8, len(effects) * 0.4 + 2))
ax.errorbar(effects, y, xerr=1.96 * ses, fmt="o", color="steelblue",
ecolor="gray", capsize=3, label="Study estimate")
ax.axvline(0, color="black", linewidth=0.8)
ax.axvline(pooled["pooled_effect"], color="red", linestyle="--",
label=f"Pooled g = {pooled['pooled_effect']:.2f}")
ax.axvspan(pooled["ci_low"], pooled["ci_high"], color="red", alpha=0.08)
ax.set_yticks(y)
ax.set_yticklabels(study_labels)
ax.set_xlabel("Effect size (Hedges' g)")
ax.set_title(f"Forest Plot (k={pooled['k_studies']}, I²={pooled['I2_percent']}%)")
ax.legend(loc="lower right", fontsize=8)
plt.tight_layout()
plt.savefig(output_path, dpi=150)
plt.close()
def funnel_plot(effects, variances, pooled_effect: float, output_path: str = "funnel_plot.png"):
"""
Funnel plot (effect size vs. standard error) for a visual publication-bias check.
Asymmetry suggests small, null-result studies may be missing from the
literature — confirm with Egger's regression test before concluding bias.
"""
ses = np.sqrt(variances)
fig, ax = plt.subplots(figsize=(7, 6))
ax.scatter(effects, ses, alpha=0.7, color="steelblue")
ax.axvline(pooled_effect, color="red", linestyle="--", label="Pooled effect")
ax.invert_yaxis()
ax.set_xlabel("Effect size")
ax.set_ylabel("Standard error")
ax.set_title("Funnel Plot")
ax.legend()
plt.tight_layout()
plt.savefig(output_path, dpi=150)
plt.close()
def eggers_test(effects, ses) -> dict:
"""
Egger's regression test for funnel plot asymmetry: regress
(effect / SE) on (1 / SE); a significant non-zero intercept suggests
small-study / publication-bias-consistent asymmetry.
"""
import statsmodels.api as sm
precision = 1 / ses
standardized_effect = effects / ses
X = sm.add_constant(precision)
fit = sm.OLS(standardized_effect, X).fit()
return {
"intercept": float(fit.params[0]),
"p_value": float(fit.pvalues[0]),
"significant_asymmetry": bool(fit.pvalues[0] < 0.05),
}
Step 4 — Moderator Analysis (Meta-Regression / Subgroups)
def subgroup_meta_analysis(df: pd.DataFrame, effect_col: str, var_col: str, group_col: str) -> pd.DataFrame:
"""
Run a separate random-effects pooled estimate within each level of a
categorical moderator (e.g., grade band, intervention type).
Args:
df: one row per study, with effect size, its variance, and a
categorical moderator column.
effect_col, var_col, group_col: column names.
Returns:
DataFrame with one row per subgroup and its pooled estimate.
"""
rows = []
for level, sub in df.groupby(group_col):
pooled = pool_random_effects(sub[effect_col].values, sub[var_col].values)
rows.append({"subgroup": level, **pooled})
return pd.DataFrame(rows)
def meta_regression(df: pd.DataFrame, effect_col: str, var_col: str, moderator_col: str) -> dict:
"""
Weighted meta-regression of effect size on a continuous moderator
(e.g., intervention duration in weeks), weighting by inverse variance.
This is a simplified fixed-effect-weighted regression; for a fully
correct random-effects meta-regression with a residual tau^2, use
`pymare` or R `metafor::rma(mods = ~ moderator)`.
"""
import statsmodels.api as sm
weights = 1 / df[var_col].values
X = sm.add_constant(df[moderator_col])
fit = sm.WLS(df[effect_col], X, weights=weights).fit()
return {
"slope": float(fit.params[moderator_col]),
"slope_se": float(fit.bse[moderator_col]),
"slope_p": float(fit.pvalues[moderator_col]),
"r_squared": float(fit.rsquared),
}
Example — Full Workflow on a Simulated Set of Studies
import numpy as np
import pandas as pd
np.random.seed(42)
n_studies = 15
# Simulate heterogeneous study-level effect sizes for a feedback intervention
true_effect = 0.35
true_tau = 0.15 # between-study SD
study_sample_sizes = np.random.randint(30, 300, n_studies)
rows = []
for i in range(n_studies):
n_per_group = study_sample_sizes[i] // 2
study_true_effect = true_effect + np.random.normal(0, true_tau)
m1, sd1 = 70 + study_true_effect * 15, 15
m2, sd2 = 70, 15
es = smd_from_means(m1, sd1, n_per_group, m2, sd2, n_per_group)
rows.append({
"study": f"Study {i+1}",
"g": es["g"], "variance": es["variance"],
"grade_band": "elementary" if i % 2 == 0 else "secondary",
"duration_weeks": np.random.randint(2, 16),
})
df_studies = pd.DataFrame(rows)
# Pool overall
pooled = pool_random_effects(df_studies["g"].values, df_studies["variance"].values)
print("Overall pooled result:", pooled)
# Forest + funnel plots
forest_plot(df_studies["study"].tolist(), df_studies["g"].values,
df_studies["variance"].values, pooled)
funnel_plot(df_studies["g"].values, df_studies["variance"].values, pooled["pooled_effect"])
# Egger's test
egger = eggers_test(df_studies["g"].values, np.sqrt(df_studies["variance"].values))
print("\nEgger's test for small-study effects:", egger)
# Subgroup analysis by grade band
subgroup_results = subgroup_meta_analysis(df_studies, "g", "variance", "grade_band")
print("\nSubgroup analysis (grade band):")
print(subgroup_results[["subgroup", "pooled_effect", "ci_low", "ci_high", "I2_percent"]])
# Meta-regression on intervention duration
reg = meta_regression(df_studies, "g", "variance", "duration_weeks")
print("\nMeta-regression on duration (weeks):", reg)
Troubleshooting
| Problem | Likely Cause | Fix |
|---|---|---|
| I² very high (>75%) but you report one pooled mean anyway | Heterogeneity not addressed | Run subgroup analysis / meta-regression; discuss heterogeneity explicitly, don't bury it |
| Negative tau² computed | Q statistic smaller than its degrees of freedom (sampling noise) | Already clamped to 0 in pool_random_effects; report tau²=0 with a note that between-study variance was not detectable |
| Funnel plot looks asymmetric with k < 10 studies | Normal with few studies; low power, not necessarily bias | State explicitly that funnel/Egger diagnostics are underpowered here |
| Effect sizes have inconsistent sign across studies | Outcome coded in opposite directions in different studies | Re-sign effects so positive always means the same substantive direction before pooling |
| Can't compute SMD from a paper | Only p-value and means reported, no SD | Request raw data from authors, or exclude the study and report why (PRISMA flow) |
Quality Checks Before Reporting a Meta-Analysis
- Document the search and screening process (databases, dates, search string, inclusion/exclusion criteria) well enough for a PRISMA flow diagram, even if you don't formally draw one.
- Re-sign every effect size to a common direction before pooling.
- Report I² and τ² alongside the pooled estimate — never report a single mean effect without a heterogeneity statistic.
- Run (or explicitly justify skipping) a publication-bias check, and say how many studies it was based on.
- If effect sizes are not independent (same sample contributing multiple outcomes), use robust variance estimation or multilevel meta-analysis rather than this skill's simple pooling function.
External Resources
- Borenstein, M., Hedges, L. V., Higgins, J. P. T., & Rothstein, H. R. (2009). Introduction to Meta-Analysis. Wiley.
- Higgins, J. P. T., et al. (2003). Measuring inconsistency in meta-analyses. BMJ, 327.
- PRISMA 2020 statement: https://www.prisma-statement.org/
- R
metaforpackage (field standard): https://www.metafor-project.org/ pymare(Python meta-analysis): https://pymare.readthedocs.io/- What Works Clearinghouse evidence standards: https://ies.ed.gov/ncee/wwc/
示例提示词
帮我汇总多篇教学干预研究的效应量,做随机效应元分析和异质性检验。
所属插件
安装该插件会一并获得同领域的 7 个 Skill / MCP。
同插件的其他科研 Skill
- 教育数据挖掘分析Mines LMS logs with Bayesian knowledge tracing, Cox dropout survival models, IRT, and sequence mining. Use to predict at-risk students or trace skill mastery.
- 教育多层线性模型Fits multilevel models for students nested in classrooms/schools, with ICC checks and growth models. Use for cluster-randomized trials or nested data.
- 教育测量项目反应理论Calibrates 2PL/3PL IRT models and plots item/test information functions and DIF. Use when calibrating test items or auditing fairness across groups.
- 教育心理学实验设计分析Designs and analyzes education experiments: RCTs, IRT scoring, growth/forgetting curves, A/B tests. Use when planning or analyzing an intervention study.
- PISA国际测评数据分析Analyzes PISA/TIMSS data with plausible-value averaging, BRR variance, and cross-national SES gradients. Use to compare country performance from assessment data.
- 量表验证与因子分析Validates survey scales with EFA/CFA, alpha/omega reliability, and measurement invariance. Use when piloting a motivation or attitude instrument.
常见问题
教育研究元分析能做什么?
Pools effect sizes across studies with random-effects models and heterogeneity stats. Use for a systematic review of intervention effects across studies。
如何安装教育研究元分析?
在科搭 Codar 左侧点击「插件」,在「组件」页签搜索「教育研究元分析」并安装;在 Claude Code 或 Codex 中可以发送本页的安装提示词,或下载 Skill 包后解压到 ~/.claude/skills 或 ~/.codex/skills。
教育研究元分析需要哪些权限?
该科研 Skill没有声明额外权限;执行具体操作时仍由 Agent 按自身规则请求确认。
教育研究元分析经过官方审核吗?
是。SpeedSkills 收录的每个版本都会记录内容摘要与权限声明,当前版本已通过安全审核。