科研 Skill

教育研究元分析

edu-meta-analysis

Pools effect sizes across studies with random-effects models and heterogeneity stats. Use for a systematic review of intervention effects across studies.

发布者 Codar安全审核通过v0.1.0更新于 2026-11-18

SKILL.md

Systematic Review and Meta-Analysis for Education Research

When to Use This Skill

Use this skill when you need to synthesize findings across multiple education studies rather than run a single new study — for example, "what is the average effect of feedback interventions on student achievement across the published literature?" or "is the effect of class-size reduction consistent across countries?" This complements the single-study skills in this plugin (learning-experiment, hierarchical-linear-modeling-edu) by operating one level up, on a collected set of effect sizes rather than raw student data.

Typical tasks:

  • Search and screen studies following a PRISMA-style workflow
  • Extract or compute a common effect size (Cohen's d / Hedges' g, or a correlation) from each study's reported statistics
  • Pool effect sizes with a random-effects (or fixed-effect) model
  • Quantify and visualize heterogeneity across studies (I², τ², Q-test)
  • Check for publication bias (funnel plot, Egger's test, trim-and-fill)
  • Explore moderators (e.g., grade level, intervention duration) via meta-regression or subgroup analysis

Do NOT use this skill to combine studies that measure conceptually different constructs just because they both report a p-value — meta- analysis assumes the pooled studies estimate a common (or at least related, explainable) underlying effect. Combining apples and oranges produces a confident-looking but meaningless pooled number.

Prerequisites

  • A set of studies identified through a documented, reproducible search (database, search string, date range, inclusion/exclusion criteria) — this skill does not perform literature search itself; pair it with database access (ERIC, PsycINFO, Web of Science, Google Scholar) or a reference manager export.
  • For each included study: enough reported statistics to compute or extract an effect size and its variance (e.g., group means + SDs + n, or a reported t/F statistic + n, or a correlation + n).
  • Python 3.10+ with numpy, pandas, scipy, matplotlib; the pymare or R metafor package is recommended for anything beyond a single simple random-effects model (subgroup meta-regression, multivariate/multilevel meta-analysis, cluster-robust variance).
  • Familiarity with the direction/sign convention for effect sizes across your included studies — a frequent, serious error is mixing studies where "positive" means opposite things (e.g., effect on dropout vs. effect on retention) without re-signing them consistently first.

Limitations

  • The pooling functions below implement a standard univariate random-effects model (DerSimonian-Laird) for independent effect sizes. They do not handle studies that contribute multiple, dependent effect sizes (e.g., several outcomes from the same sample) — that requires robust variance estimation (RVE) or multilevel meta-analysis, which is not implemented here; use R metafor/robumeta or Python pymare for that case, and do not naively pool dependent effects as if independent.
  • Fixed-effect vs. random-effects is a substantive choice, not just a technical default: fixed-effect assumes one true effect underlies every study, which is rarely plausible in education research with varying populations, interventions, and settings. This skill defaults to random-effects; switch only with a clear justification.
  • Publication bias diagnostics (funnel plot asymmetry, Egger's test) are low-powered with fewer than ~10 studies and are suggestive, not definitive — do not present them as proof of (or exoneration from) publication bias with a small set of studies.
  • This skill does not conduct systematic search or screening itself, and does not substitute for a PRISMA flow diagram or a pre-registered protocol (e.g., in PROSPERO) when one is required by your target venue.

Environment Setup

pip install numpy pandas scipy matplotlib
# Recommended for subgroup/meta-regression and robust variance estimation:
pip install pymare
# R alternative (field standard for complex designs):
# install.packages(c("metafor", "robumeta"))

Step 1 — Compute Effect Sizes from Common Reported Statistics

import numpy as np
import pandas as pd

def smd_from_means(m1, sd1, n1, m2, sd2, n2) -> dict:
    """
    Standardized mean difference (Cohen's d) with small-sample correction
    (Hedges' g) and its variance, from two group means/SDs/sizes.

    Args:
        m1, sd1, n1: treatment group mean, SD, sample size.
        m2, sd2, n2: control group mean, SD, sample size.

    Returns:
        dict with d, g, se, variance, n_total.
    """
    pooled_sd = np.sqrt(((n1 - 1) * sd1**2 + (n2 - 1) * sd2**2) / (n1 + n2 - 2))
    d = (m1 - m2) / pooled_sd

    # Hedges' small-sample correction factor
    df = n1 + n2 - 2
    j = 1 - 3 / (4 * df - 1)
    g = d * j

    var_d = (n1 + n2) / (n1 * n2) + d**2 / (2 * (n1 + n2))
    var_g = (j**2) * var_d

    return {"d": d, "g": g, "se": np.sqrt(var_g), "variance": var_g, "n_total": n1 + n2}


def smd_from_t_stat(t_stat, n1, n2) -> dict:
    """Convert an independent-samples t-statistic to Hedges' g."""
    d = t_stat * np.sqrt(1 / n1 + 1 / n2)
    df = n1 + n2 - 2
    j = 1 - 3 / (4 * df - 1)
    g = d * j
    var_g = (j**2) * ((n1 + n2) / (n1 * n2) + d**2 / (2 * (n1 + n2)))
    return {"d": d, "g": g, "se": np.sqrt(var_g), "variance": var_g, "n_total": n1 + n2}


def r_to_z(r: float, n: int) -> dict:
    """Fisher's z transformation of a correlation, with its variance (1/(n-3))."""
    z = 0.5 * np.log((1 + r) / (1 - r))
    var_z = 1 / (n - 3)
    return {"z": z, "se": np.sqrt(var_z), "variance": var_z, "n": n}


def z_to_r(z: float) -> float:
    """Back-transform Fisher's z to a correlation for reporting."""
    return (np.exp(2 * z) - 1) / (np.exp(2 * z) + 1)

Step 2 — Random-Effects Pooling (DerSimonian-Laird)

def pool_random_effects(effect_sizes: np.ndarray, variances: np.ndarray) -> dict:
    """
    DerSimonian-Laird random-effects meta-analysis.

    Args:
        effect_sizes: array of study effect sizes (e.g., Hedges' g per study).
        variances: array of each effect size's sampling variance.

    Returns:
        dict with pooled_effect, se, ci_low, ci_high, tau2, Q, I2, p_heterogeneity.
    """
    k = len(effect_sizes)
    weights_fixed = 1 / variances

    # Fixed-effect pooled estimate (needed for Q statistic)
    pooled_fixed = np.sum(weights_fixed * effect_sizes) / np.sum(weights_fixed)
    Q = np.sum(weights_fixed * (effect_sizes - pooled_fixed) ** 2)
    df = k - 1

    from scipy.stats import chi2
    p_heterogeneity = 1 - chi2.cdf(Q, df) if df > 0 else np.nan

    # Tau^2 (between-study variance) via DerSimonian-Laird method of moments
    c = np.sum(weights_fixed) - np.sum(weights_fixed**2) / np.sum(weights_fixed)
    tau2 = max(0.0, (Q - df) / c) if c > 0 else 0.0

    # Random-effects weights and pooled estimate
    weights_re = 1 / (variances + tau2)
    pooled_re = np.sum(weights_re * effect_sizes) / np.sum(weights_re)
    se_re = np.sqrt(1 / np.sum(weights_re))

    i2 = max(0.0, (Q - df) / Q) * 100 if Q > 0 else 0.0

    return {
        "k_studies": k,
        "pooled_effect": round(pooled_re, 4),
        "se": round(se_re, 4),
        "ci_low": round(pooled_re - 1.96 * se_re, 4),
        "ci_high": round(pooled_re + 1.96 * se_re, 4),
        "tau2": round(tau2, 4),
        "Q": round(Q, 3),
        "df": df,
        "I2_percent": round(i2, 1),
        "p_heterogeneity": round(p_heterogeneity, 4) if not np.isnan(p_heterogeneity) else None,
    }

I² interpretation (Higgins et al., 2003): ~25% low, ~50% moderate, ~75% high heterogeneity. High I² means a single pooled number may be misleading — look for moderators (Step 4) rather than reporting one average effect.

Step 3 — Forest Plot and Funnel Plot

import matplotlib.pyplot as plt

def forest_plot(study_labels, effects, variances, pooled: dict, output_path: str = "forest_plot.png"):
    """
    Forest plot of individual study effects and the pooled random-effects estimate.
    """
    ses = np.sqrt(variances)
    y = np.arange(len(effects))[::-1]

    fig, ax = plt.subplots(figsize=(8, len(effects) * 0.4 + 2))
    ax.errorbar(effects, y, xerr=1.96 * ses, fmt="o", color="steelblue",
                ecolor="gray", capsize=3, label="Study estimate")
    ax.axvline(0, color="black", linewidth=0.8)
    ax.axvline(pooled["pooled_effect"], color="red", linestyle="--",
               label=f"Pooled g = {pooled['pooled_effect']:.2f}")
    ax.axvspan(pooled["ci_low"], pooled["ci_high"], color="red", alpha=0.08)

    ax.set_yticks(y)
    ax.set_yticklabels(study_labels)
    ax.set_xlabel("Effect size (Hedges' g)")
    ax.set_title(f"Forest Plot (k={pooled['k_studies']}, I²={pooled['I2_percent']}%)")
    ax.legend(loc="lower right", fontsize=8)
    plt.tight_layout()
    plt.savefig(output_path, dpi=150)
    plt.close()


def funnel_plot(effects, variances, pooled_effect: float, output_path: str = "funnel_plot.png"):
    """
    Funnel plot (effect size vs. standard error) for a visual publication-bias check.
    Asymmetry suggests small, null-result studies may be missing from the
    literature — confirm with Egger's regression test before concluding bias.
    """
    ses = np.sqrt(variances)
    fig, ax = plt.subplots(figsize=(7, 6))
    ax.scatter(effects, ses, alpha=0.7, color="steelblue")
    ax.axvline(pooled_effect, color="red", linestyle="--", label="Pooled effect")
    ax.invert_yaxis()
    ax.set_xlabel("Effect size")
    ax.set_ylabel("Standard error")
    ax.set_title("Funnel Plot")
    ax.legend()
    plt.tight_layout()
    plt.savefig(output_path, dpi=150)
    plt.close()


def eggers_test(effects, ses) -> dict:
    """
    Egger's regression test for funnel plot asymmetry: regress
    (effect / SE) on (1 / SE); a significant non-zero intercept suggests
    small-study / publication-bias-consistent asymmetry.
    """
    import statsmodels.api as sm
    precision = 1 / ses
    standardized_effect = effects / ses
    X = sm.add_constant(precision)
    fit = sm.OLS(standardized_effect, X).fit()
    return {
        "intercept": float(fit.params[0]),
        "p_value": float(fit.pvalues[0]),
        "significant_asymmetry": bool(fit.pvalues[0] < 0.05),
    }

Step 4 — Moderator Analysis (Meta-Regression / Subgroups)

def subgroup_meta_analysis(df: pd.DataFrame, effect_col: str, var_col: str, group_col: str) -> pd.DataFrame:
    """
    Run a separate random-effects pooled estimate within each level of a
    categorical moderator (e.g., grade band, intervention type).

    Args:
        df: one row per study, with effect size, its variance, and a
            categorical moderator column.
        effect_col, var_col, group_col: column names.

    Returns:
        DataFrame with one row per subgroup and its pooled estimate.
    """
    rows = []
    for level, sub in df.groupby(group_col):
        pooled = pool_random_effects(sub[effect_col].values, sub[var_col].values)
        rows.append({"subgroup": level, **pooled})
    return pd.DataFrame(rows)


def meta_regression(df: pd.DataFrame, effect_col: str, var_col: str, moderator_col: str) -> dict:
    """
    Weighted meta-regression of effect size on a continuous moderator
    (e.g., intervention duration in weeks), weighting by inverse variance.

    This is a simplified fixed-effect-weighted regression; for a fully
    correct random-effects meta-regression with a residual tau^2, use
    `pymare` or R `metafor::rma(mods = ~ moderator)`.
    """
    import statsmodels.api as sm
    weights = 1 / df[var_col].values
    X = sm.add_constant(df[moderator_col])
    fit = sm.WLS(df[effect_col], X, weights=weights).fit()
    return {
        "slope": float(fit.params[moderator_col]),
        "slope_se": float(fit.bse[moderator_col]),
        "slope_p": float(fit.pvalues[moderator_col]),
        "r_squared": float(fit.rsquared),
    }

Example — Full Workflow on a Simulated Set of Studies

import numpy as np
import pandas as pd

np.random.seed(42)
n_studies = 15

# Simulate heterogeneous study-level effect sizes for a feedback intervention
true_effect = 0.35
true_tau = 0.15  # between-study SD
study_sample_sizes = np.random.randint(30, 300, n_studies)

rows = []
for i in range(n_studies):
    n_per_group = study_sample_sizes[i] // 2
    study_true_effect = true_effect + np.random.normal(0, true_tau)
    m1, sd1 = 70 + study_true_effect * 15, 15
    m2, sd2 = 70, 15
    es = smd_from_means(m1, sd1, n_per_group, m2, sd2, n_per_group)
    rows.append({
        "study": f"Study {i+1}",
        "g": es["g"], "variance": es["variance"],
        "grade_band": "elementary" if i % 2 == 0 else "secondary",
        "duration_weeks": np.random.randint(2, 16),
    })

df_studies = pd.DataFrame(rows)

# Pool overall
pooled = pool_random_effects(df_studies["g"].values, df_studies["variance"].values)
print("Overall pooled result:", pooled)

# Forest + funnel plots
forest_plot(df_studies["study"].tolist(), df_studies["g"].values,
            df_studies["variance"].values, pooled)
funnel_plot(df_studies["g"].values, df_studies["variance"].values, pooled["pooled_effect"])

# Egger's test
egger = eggers_test(df_studies["g"].values, np.sqrt(df_studies["variance"].values))
print("\nEgger's test for small-study effects:", egger)

# Subgroup analysis by grade band
subgroup_results = subgroup_meta_analysis(df_studies, "g", "variance", "grade_band")
print("\nSubgroup analysis (grade band):")
print(subgroup_results[["subgroup", "pooled_effect", "ci_low", "ci_high", "I2_percent"]])

# Meta-regression on intervention duration
reg = meta_regression(df_studies, "g", "variance", "duration_weeks")
print("\nMeta-regression on duration (weeks):", reg)

Troubleshooting

ProblemLikely CauseFix
I² very high (>75%) but you report one pooled mean anywayHeterogeneity not addressedRun subgroup analysis / meta-regression; discuss heterogeneity explicitly, don't bury it
Negative tau² computedQ statistic smaller than its degrees of freedom (sampling noise)Already clamped to 0 in pool_random_effects; report tau²=0 with a note that between-study variance was not detectable
Funnel plot looks asymmetric with k < 10 studiesNormal with few studies; low power, not necessarily biasState explicitly that funnel/Egger diagnostics are underpowered here
Effect sizes have inconsistent sign across studiesOutcome coded in opposite directions in different studiesRe-sign effects so positive always means the same substantive direction before pooling
Can't compute SMD from a paperOnly p-value and means reported, no SDRequest raw data from authors, or exclude the study and report why (PRISMA flow)

Quality Checks Before Reporting a Meta-Analysis

  1. Document the search and screening process (databases, dates, search string, inclusion/exclusion criteria) well enough for a PRISMA flow diagram, even if you don't formally draw one.
  2. Re-sign every effect size to a common direction before pooling.
  3. Report I² and τ² alongside the pooled estimate — never report a single mean effect without a heterogeneity statistic.
  4. Run (or explicitly justify skipping) a publication-bias check, and say how many studies it was based on.
  5. If effect sizes are not independent (same sample contributing multiple outcomes), use robust variance estimation or multilevel meta-analysis rather than this skill's simple pooling function.

External Resources

示例提示词

帮我汇总多篇教学干预研究的效应量,做随机效应元分析和异质性检验。

所属插件

教育学面向教育研究与教学:教育数据挖掘与学习分析、IRT 测量、学习实验设计、PISA 国际测评数据,以及中学各学科辅导。

安装该插件会一并获得同领域的 7 个 Skill / MCP。

同插件的其他科研 Skill

常见问题

教育研究元分析能做什么?

Pools effect sizes across studies with random-effects models and heterogeneity stats. Use for a systematic review of intervention effects across studies。

如何安装教育研究元分析?

在科搭 Codar 左侧点击「插件」,在「组件」页签搜索「教育研究元分析」并安装;在 Claude Code 或 Codex 中可以发送本页的安装提示词,或下载 Skill 包后解压到 ~/.claude/skills 或 ~/.codex/skills。

教育研究元分析需要哪些权限?

该科研 Skill没有声明额外权限;执行具体操作时仍由 Agent 按自身规则请求确认。

教育研究元分析经过官方审核吗?

是。SpeedSkills 收录的每个版本都会记录内容摘要与权限声明,当前版本已通过安全审核。