科研 Skill
统计假设检定选择
statistical-testing
Selects and runs the correct hypothesis test for given data properties, covering parametric, non-parametric tests, effect sizes, and multiple-comparison correction.
SKILL.md
Statistical Testing
Choose and execute the right statistical test for your data. This skill covers normality checks, parametric and non-parametric tests, effect size computation, and multiple comparison correction.
Decision Tree
| Scenario | Groups | Normal? | Recommended Test |
|---|---|---|---|
| Compare means | 2 independent | Yes | Independent t-test |
| Compare means | 2 independent | No | Mann-Whitney U |
| Compare means | 2 paired | Yes | Paired t-test |
| Compare means | 2 paired | No | Wilcoxon signed-rank |
| Compare means | ≥3 independent | Yes | One-way ANOVA |
| Compare means | ≥3 independent | No | Kruskal-Wallis |
| Compare means | ≥3 repeated | Yes | Repeated-measures ANOVA |
| Compare means | ≥3 repeated | No | Friedman test |
| Association | 2 categorical | Expected ≥5 | Chi-square |
| Association | 2 categorical | Expected <5 | Fisher exact |
| Correlation | 2 continuous | Both normal | Pearson r |
| Correlation | 2 continuous | Non-normal | Spearman ρ |
Setup
pip install scipy numpy pandas statsmodels pingouin
Core Implementation
"""
statistical_testing.py
Complete statistical testing toolkit with automatic test selection.
"""
import warnings
import numpy as np
import pandas as pd
from scipy import stats
from statsmodels.stats import multitest
from statsmodels.stats.power import TTestIndPower, FTestAnovaPower
from typing import Union, Optional, Tuple, List, Dict, Any
# ─────────────────────────────────────────────
# 1. Normality Tests
# ─────────────────────────────────────────────
def test_normality(
data: np.ndarray,
alpha: float = 0.05,
verbose: bool = True,
) -> Dict[str, Any]:
"""
Run Shapiro-Wilk, Kolmogorov-Smirnov, and D'Agostino-Pearson normality tests.
Parameters
----------
data : array-like
1-D numeric data.
alpha : float
Significance level.
verbose : bool
Print a summary.
Returns
-------
dict with keys: shapiro, ks, dagostino, is_normal
"""
data = np.asarray(data, dtype=float)
data = data[~np.isnan(data)]
n = len(data)
results: Dict[str, Any] = {}
# Shapiro-Wilk (best for n < 5000)
if n < 5000:
stat_sw, p_sw = stats.shapiro(data)
results["shapiro"] = {"statistic": stat_sw, "p_value": p_sw, "normal": p_sw > alpha}
else:
results["shapiro"] = {"statistic": None, "p_value": None, "normal": None,
"note": "n>=5000; skipped Shapiro-Wilk"}
# Kolmogorov-Smirnov against fitted normal
mu, sigma = np.mean(data), np.std(data, ddof=1)
stat_ks, p_ks = stats.kstest(data, "norm", args=(mu, sigma))
results["ks"] = {"statistic": stat_ks, "p_value": p_ks, "normal": p_ks > alpha}
# D'Agostino-Pearson (needs n >= 20)
if n >= 20:
stat_da, p_da = stats.normaltest(data)
results["dagostino"] = {"statistic": stat_da, "p_value": p_da, "normal": p_da > alpha}
else:
results["dagostino"] = {"statistic": None, "p_value": None, "normal": None,
"note": "n<20; skipped D'Agostino"}
# Consensus: normal if at least 2 out of 3 available tests agree
votes = [v["normal"] for v in results.values() if isinstance(v, dict) and v["normal"] is not None]
results["is_normal"] = sum(votes) >= (len(votes) / 2)
results["n"] = n
if verbose:
print(f"Normality tests (n={n}, alpha={alpha})")
for test_name, res in results.items():
if isinstance(res, dict) and "p_value" in res and res["p_value"] is not None:
verdict = "NORMAL" if res["normal"] else "NON-NORMAL"
print(f" {test_name:15s}: stat={res['statistic']:.4f}, p={res['p_value']:.4f} [{verdict}]")
print(f" Consensus: {'NORMAL' if results['is_normal'] else 'NON-NORMAL'}\n")
return results
# ─────────────────────────────────────────────
# 2. Effect Sizes
# ─────────────────────────────────────────────
def cohens_d(group1: np.ndarray, group2: np.ndarray) -> float:
"""Cohen's d for two independent groups."""
g1, g2 = np.asarray(group1), np.asarray(group2)
n1, n2 = len(g1), len(g2)
pooled_sd = np.sqrt(((n1 - 1) * np.var(g1, ddof=1) + (n2 - 1) * np.var(g2, ddof=1)) / (n1 + n2 - 2))
return (np.mean(g1) - np.mean(g2)) / pooled_sd
def eta_squared(f_statistic: float, df_between: int, df_within: int) -> float:
"""Eta-squared from F-statistic for one-way ANOVA."""
ss_between = f_statistic * df_between
ss_total = ss_between + df_within
return ss_between / ss_total
def cramers_v(contingency_table: np.ndarray) -> float:
"""Cramér's V for association in a contingency table."""
chi2 = stats.chi2_contingency(contingency_table, correction=False)[0]
n = contingency_table.sum()
min_dim = min(contingency_table.shape) - 1
return np.sqrt(chi2 / (n * min_dim))
def interpret_effect_size(d: float, measure: str = "cohens_d") -> str:
"""Return a qualitative label for an effect size."""
thresholds = {
"cohens_d": [(0.2, "small"), (0.5, "medium"), (0.8, "large")],
"eta_squared": [(0.01, "small"), (0.06, "medium"), (0.14, "large")],
"cramers_v": [(0.1, "small"), (0.3, "medium"), (0.5, "large")],
}
d = abs(d)
for threshold, label in thresholds.get(measure, []):
if d < threshold:
return label
return "large"
# ─────────────────────────────────────────────
# 3. Automatic Test Selection
# ─────────────────────────────────────────────
def run_comparison(
*groups: np.ndarray,
paired: bool = False,
alpha: float = 0.05,
verbose: bool = True,
) -> Dict[str, Any]:
"""
Automatically select and run the appropriate comparison test.
Parameters
----------
*groups : array-like
Two or more numeric arrays (one per group).
paired : bool
Whether observations are paired / repeated measures.
alpha : float
Significance level.
verbose : bool
Print result summary.
Returns
-------
dict with keys: test_name, statistic, p_value, effect_size, significant
"""
groups = [np.asarray(g, dtype=float) for g in groups]
k = len(groups)
if k < 2:
raise ValueError("Need at least 2 groups.")
# Check normality for each group
all_normal = all(test_normality(g, alpha=alpha, verbose=False)["is_normal"] for g in groups)
result: Dict[str, Any] = {}
if k == 2:
g1, g2 = groups[0], groups[1]
if paired:
if all_normal:
stat, p = stats.ttest_rel(g1, g2)
result["test_name"] = "Paired t-test"
d = np.mean(g1 - g2) / np.std(g1 - g2, ddof=1)
result["effect_size"] = {"cohens_d": round(d, 4)}
else:
stat, p = stats.wilcoxon(g1, g2)
result["test_name"] = "Wilcoxon signed-rank"
result["effect_size"] = {"rank_biserial": round(1 - (2 * stat) / (len(g1) * (len(g1) + 1)), 4)}
else:
# Levene's test for equal variances
_, p_levene = stats.levene(g1, g2)
equal_var = p_levene > alpha
if all_normal:
stat, p = stats.ttest_ind(g1, g2, equal_var=equal_var)
result["test_name"] = f"Independent t-test ({'equal' if equal_var else 'Welch'} variance)"
d = cohens_d(g1, g2)
result["effect_size"] = {"cohens_d": round(d, 4),
"interpretation": interpret_effect_size(d)}
else:
stat, p = stats.mannwhitneyu(g1, g2, alternative="two-sided")
result["test_name"] = "Mann-Whitney U"
n1, n2 = len(g1), len(g2)
rb = 1 - (2 * stat) / (n1 * n2)
result["effect_size"] = {"rank_biserial_r": round(rb, 4)}
else:
# k >= 3
if paired:
if all_normal:
# Repeated-measures via pingouin if available, else warn
try:
import pingouin as pg
df_long = pd.DataFrame({
"value": np.concatenate(groups),
"group": np.repeat(np.arange(k), [len(g) for g in groups]),
"subject": np.tile(np.arange(len(groups[0])), k),
})
aov = pg.rm_anova(data=df_long, dv="value", within="group", subject="subject")
stat = aov["F"].iloc[0]
p = aov["p-unc"].iloc[0]
result["test_name"] = "Repeated-measures ANOVA (pingouin)"
result["effect_size"] = {"eta_squared": round(aov["np2"].iloc[0], 4)}
except ImportError:
warnings.warn("pingouin not installed; falling back to Friedman test.")
stat, p = stats.friedmanchisquare(*groups)
result["test_name"] = "Friedman test (fallback)"
result["effect_size"] = {}
else:
stat, p = stats.friedmanchisquare(*groups)
result["test_name"] = "Friedman test"
result["effect_size"] = {}
else:
if all_normal:
stat, p = stats.f_oneway(*groups)
result["test_name"] = "One-way ANOVA"
total_n = sum(len(g) for g in groups)
df_between = k - 1
df_within = total_n - k
es = eta_squared(stat, df_between, df_within)
result["effect_size"] = {"eta_squared": round(es, 4),
"interpretation": interpret_effect_size(es, "eta_squared")}
else:
stat, p = stats.kruskal(*groups)
result["test_name"] = "Kruskal-Wallis H"
result["effect_size"] = {}
result["statistic"] = round(float(stat), 4)
result["p_value"] = round(float(p), 6)
result["significant"] = p < alpha
result["alpha"] = alpha
if verbose:
print(f"Test: {result['test_name']}")
print(f" Statistic = {result['statistic']}, p = {result['p_value']}")
print(f" Significant at alpha={alpha}: {result['significant']}")
if result.get("effect_size"):
print(f" Effect size: {result['effect_size']}")
print()
return result
# ─────────────────────────────────────────────
# 4. Multiple Comparison Correction
# ─────────────────────────────────────────────
def correct_pvalues(
p_values: List[float],
method: str = "fdr_bh",
alpha: float = 0.05,
) -> pd.DataFrame:
"""
Apply multiple comparison correction.
Parameters
----------
p_values : list of float
method : str
One of: bonferroni, holm, fdr_bh (Benjamini-Hochberg),
fdr_by (Benjamini-Yekutieli), sidak
alpha : float
Returns
-------
DataFrame with original p-values, corrected p-values, and rejection flags.
"""
reject, p_corrected, _, _ = multitest.multipletests(p_values, alpha=alpha, method=method)
return pd.DataFrame({
"p_original": p_values,
"p_corrected": p_corrected,
"reject_H0": reject,
"method": method,
})
# ─────────────────────────────────────────────
# 5. Power Analysis
# ─────────────────────────────────────────────
def power_analysis_ttest(
effect_size: float = 0.5,
alpha: float = 0.05,
power: float = 0.80,
n_per_group: Optional[int] = None,
) -> Dict[str, float]:
"""
Two-sample t-test power analysis.
Provide any three of (effect_size, alpha, power, n_per_group) to solve for the fourth.
"""
analysis = TTestIndPower()
if n_per_group is None:
n = analysis.solve_power(effect_size=effect_size, alpha=alpha, power=power, ratio=1.0)
return {"required_n_per_group": np.ceil(n), "effect_size": effect_size,
"alpha": alpha, "power": power}
else:
achieved_power = analysis.solve_power(effect_size=effect_size, alpha=alpha,
nobs1=n_per_group, ratio=1.0)
return {"n_per_group": n_per_group, "effect_size": effect_size,
"alpha": alpha, "achieved_power": round(achieved_power, 4)}
def chi_square_test_with_effect(
contingency: np.ndarray,
alpha: float = 0.05,
) -> Dict[str, Any]:
"""Chi-square test with Cramér's V effect size."""
chi2, p, dof, expected = stats.chi2_contingency(contingency)
v = cramers_v(contingency)
result = {
"test_name": "Chi-square" if expected.min() >= 5 else "Fisher exact (recommended)",
"chi2": round(chi2, 4),
"p_value": round(p, 6),
"dof": dof,
"cramers_v": round(v, 4),
"significant": p < alpha,
}
if expected.min() < 5:
if contingency.shape == (2, 2):
_, p_fisher = stats.fisher_exact(contingency)
result["fisher_p"] = round(p_fisher, 6)
return result
Example 1 — Comparing Three Treatment Groups
import numpy as np
from statistical_testing import run_comparison, correct_pvalues, power_analysis_ttest
rng = np.random.default_rng(42)
# Simulate three treatment groups (non-normal distributions)
control = rng.exponential(scale=5, size=40)
treatment1 = rng.exponential(scale=7, size=38)
treatment2 = rng.exponential(scale=9, size=42)
# Automatic test selection
result = run_comparison(control, treatment1, treatment2, alpha=0.05)
# → Kruskal-Wallis H (non-normal data detected automatically)
# If significant, run post-hoc pairwise tests with FDR correction
from scipy.stats import mannwhitneyu
pairs = [
("control vs t1", mannwhitneyu(control, treatment1, alternative="two-sided").pvalue),
("control vs t2", mannwhitneyu(control, treatment2, alternative="two-sided").pvalue),
("t1 vs t2", mannwhitneyu(treatment1, treatment2, alternative="two-sided").pvalue),
]
labels, raw_p = zip(*pairs)
correction_df = correct_pvalues(list(raw_p), method="fdr_bh")
correction_df.index = labels
print(correction_df)
Example 2 — Two-Group Comparison with Power Report
import numpy as np
from statistical_testing import (
test_normality, run_comparison, cohens_d, power_analysis_ttest
)
rng = np.random.default_rng(0)
pre = rng.normal(loc=100, scale=15, size=30)
post = pre + rng.normal(loc=8, scale=10, size=30) # ~0.5 SD improvement
# Step 1: Check normality of the difference
diff = post - pre
norm_result = test_normality(diff)
# Step 2: Run the appropriate test
result = run_comparison(pre, post, paired=True)
# Step 3: Retrospective power
d = result["effect_size"].get("cohens_d", cohens_d(pre, post))
power_report = power_analysis_ttest(effect_size=abs(d), alpha=0.05, n_per_group=30)
print(f"Achieved power: {power_report['achieved_power']:.2f}")
# Step 4: How many participants for 90% power?
needed = power_analysis_ttest(effect_size=abs(d), alpha=0.05, power=0.90)
print(f"N needed for 90% power: {int(needed['required_n_per_group'])}")
Categorical Data Example
import numpy as np
from statistical_testing import chi_square_test_with_effect, correct_pvalues
# 2x2 contingency: treatment vs outcome
table = np.array([[45, 15],
[30, 30]])
result = chi_square_test_with_effect(table)
print(result)
# {'test_name': 'Chi-square', 'chi2': ..., 'p_value': ..., 'cramers_v': ..., 'significant': True}
# 3x3 contingency across three centres
multi_table = np.array([[50, 20, 10],
[40, 25, 15],
[35, 30, 20]])
result3 = chi_square_test_with_effect(multi_table)
print(result3)
Quick Reference: Effect Size Benchmarks
| Measure | Small | Medium | Large |
|---|---|---|---|
| Cohen's d | 0.2 | 0.5 | 0.8 |
| η² (eta-squared) | 0.01 | 0.06 | 0.14 |
| Cramér's V (2×2) | 0.1 | 0.3 | 0.5 |
| Pearson r | 0.1 | 0.3 | 0.5 |
Multiple Comparison Methods
| Method | Controls | Best For |
|---|---|---|
| Bonferroni | Family-wise error rate (FWER) | Few comparisons, strict control |
| Holm | FWER (less conservative) | General use |
| Benjamini-Hochberg (FDR) | False discovery rate | Many comparisons (genomics, etc.) |
| Benjamini-Yekutieli | FDR under dependence | Correlated tests |
Common Pitfalls
- Assuming normality without testing: Always run normality checks, especially for n < 30.
- Ignoring equal-variance assumption: Run Levene's test before independent t-test.
- Multiple comparisons inflation: Any time you run ≥3 tests on the same dataset, apply correction.
- Over-relying on p < 0.05: Always report effect sizes alongside p-values.
- Wrong test for paired data: Pre/post measurements are paired; use paired tests.
Environment Variables
No API keys required. All computation is local.
# Optional: set random seed for reproducibility in scripts
export STATS_RANDOM_SEED=42
示例提示词
帮我根据数据特征选出合适的假设检验方法并校正多重比较。
所属插件
安装该插件会一并获得同领域的 54 个 Skill / MCP。
同插件的其他科研 Skill
- 智能体资源节约守则Applies token- and context-saving habits for tool choice, PDF handling, persistent memory, and privacy checks. Use at the start of sessions with many files or PDFs, or multi-session work.
- 英文论文双语精读器把英文论文转成中英对照精读档并生成离线双语阅读器,标注要点并输出读书笔记。精读单篇英文论文、要中英对照阅读时使用。
- 研究方法比较论证Compares quantitative, qualitative, and mixed research methods against what each can and cannot answer to build a methods table. Use when justifying a method choice in a thesis.
- 复杂系统分析方法Analyzes complex systems with power-law fitting, Hurst exponent, fractal dimension, agent-based modeling, and entropy measures. Use for nonlinear or complex-system data.
- 研究环境容器化部署Builds Docker and Singularity containers for reproducible computation and HPC deployment of research environments. Use when packaging an analysis pipeline for reuse or HPC.
- DVC数据版本管理Sets up DVC for dataset and pipeline versioning, remote storage, and experiment comparison in reproducible ML research workflows. Use when tracking large datasets or ML experiment runs.
- 期刊级统计图表绘制Creates publication-quality multi-panel figures with matplotlib, seaborn, or ggplot2 using colorblind-safe palettes and journal export settings. Use when preparing figures for a paper.
- 综述模块拆解规划Breaks a refined literature-review question into 3-5 non-overlapping modules to build a reading map, outline, or search plan. Use when structuring a literature review.
- 学位论文答辩教练模拟学位口试委员提问并提供承认回应证据三段式答辩框架与模拟问答。准备论文口试或研讨会问答环节时使用。
- 实验设计与样本量规划Designs rigorous experiments covering sample-size calculation, randomization strategy, pre-registration, and A/B test duration estimation. Use when planning an experiment or A/B test.
- 管理实验设计架构为管理与商学情境实验规划组间组内设计、对抗平衡与操弄检核,并产出设计书与情境材料模板。做商学实验设计时使用。
- 科研经费申请撰写Structures research grant proposals including NSF/NIH Specific Aims, budget justification, ERC narrative sections, and biosketch formatting. Use when drafting a funding application.
- 人类可读文档撰写规范Guides writing human-facing docs like README, PLAN, survey notes, and reviewer-response drafts with a clear two-tier structure. Use for text a human collaborator will read.
- 知识图谱与SPARQL检索Builds knowledge graphs and runs SPARQL queries over RDF and Wikidata for entity linking in research and library data. Use when modeling research entities as a linked graph.
- LaTeX论文写作与排版Writes, compiles, and submits LaTeX papers covering IMRaD structure, key packages, bibliography management, and arXiv preparation. Use when authoring or debugging a LaTeX manuscript.
- 文献比较矩阵构建器把一批PDF整理成含撤稿查核与APA7书目的文献资料库,并生成横向比较矩阵表。整理综述前要比较文献时使用。
- 跨库学术文献检索Searches OpenAlex, Semantic Scholar, and arXiv for academic literature with deduplication, citation snowballing, and BibTeX export. Use when collecting papers for a review or topic.
- 文献调研语料库维护Finds, verifies, and maintains a literature corpus from arXiv, OpenAlex, Crossref, and related databases with BibTeX entries and per-paper notes. Use when sourcing references for a paper.
- 文献综述研究缺口定位Converts a literature-review synthesis into a concrete, defensible research gap and future-research direction. Use when writing a conclusion or thesis contribution section.
- 文献矛盾结论梳理Analyzes conflicting findings across papers and explains disagreement by sample, method, or context differences for a stronger synthesis. Use when review sources disagree.
- 效应量汇总元分析Pools effect sizes across studies with fixed and random-effects models, heterogeneity tests, forest and funnel plots, and subgroup analysis. Use to synthesize quantitative study results.
- 缺失数据处理Diagnoses missing-data mechanisms such as MCAR, MAR, or MNAR and applies multiple imputation or deletion with sensitivity analysis. Use when a dataset has missing values.
- 混合方法研究设计Designs mixed-methods studies integrating qualitative coding, triangulation, content analysis, and QCA with inter-rater reliability checks. Use when combining qual and quant data.
- 科研机器学习流程Applies machine learning to research questions with scikit-learn pipelines, AutoML, SHAP explainability, and nested cross-validation. Use when building a predictive model for a study.
常见问题
统计假设检定选择能做什么?
Selects and runs the correct hypothesis test for given data properties, covering parametric, non-parametric tests, effect sizes, and multiple-comparison correction。
如何安装统计假设检定选择?
在科搭 Codar 左侧点击「插件」,在「组件」页签搜索「统计假设检定选择」并安装;在 Claude Code 或 Codex 中可以发送本页的安装提示词,或下载 Skill 包后解压到 ~/.claude/skills 或 ~/.codex/skills。
统计假设检定选择需要哪些权限?
该科研 Skill没有声明额外权限;执行具体操作时仍由 Agent 按自身规则请求确认。
统计假设检定选择经过官方审核吗?
是。SpeedSkills 收录的每个版本都会记录内容摘要与权限声明,当前版本已通过安全审核。