科研 Skill
实验设计与样本量规划
experimental-design
Designs rigorous experiments covering sample-size calculation, randomization strategy, pre-registration, and A/B test duration estimation. Use when planning an experiment or A/B test.
SKILL.md
Experimental Design
Plan experiments with correct statistical power, principled randomization, and transparent pre-registration before data collection begins.
Key Concepts
| Term | Definition |
|---|---|
| α (alpha) | Type I error rate — false positive probability (typically 0.05) |
| β (beta) | Type II error rate — false negative probability (typically 0.20) |
| Power (1−β) | Probability of detecting a true effect (typically 0.80 or 0.90) |
| MDES | Minimum Detectable Effect Size — smallest effect your design can detect |
| ICC | Intra-cluster correlation — needed for cluster-randomised trials |
Setup
pip install statsmodels numpy pandas scipy osfclient
Core Implementation
"""
experimental_design.py
Sample size, randomization, and pre-registration utilities.
"""
import hashlib
import math
import os
import warnings
from typing import Dict, List, Optional, Tuple, Union
import numpy as np
import pandas as pd
from scipy import stats
from statsmodels.stats.power import (
TTestIndPower,
TTestPower,
FTestAnovaPower,
NormalIndPower,
)
from statsmodels.stats.proportion import proportion_effectsize
# ─────────────────────────────────────────────
# 1. Sample Size Calculation
# ─────────────────────────────────────────────
def calculate_sample_size(
effect_size: Optional[float] = None,
alpha: float = 0.05,
power: float = 0.80,
test_type: str = "two-sample-t",
ratio: float = 1.0,
k_groups: int = 2,
p1: Optional[float] = None,
p2: Optional[float] = None,
two_tailed: bool = True,
verbose: bool = True,
) -> Dict[str, float]:
"""
Calculate required sample size for common experimental designs.
Parameters
----------
effect_size : float, optional
Standardised effect size (Cohen's d, f, h, w).
Required unless p1 and p2 are provided.
alpha : float
Type I error rate.
power : float
Desired statistical power (1 - beta).
test_type : str
One of: 'two-sample-t', 'paired-t', 'one-sample-t',
'anova', 'proportion-z', 'proportion-chi2'
ratio : float
n_group2 / n_group1 (for unequal allocation).
k_groups : int
Number of groups (for ANOVA only).
p1, p2 : float, optional
Event rates for proportion tests (auto-computes Cohen's h).
two_tailed : bool
Whether to use two-tailed test.
Returns
-------
dict with n_per_group, total_n, effect_size, alpha, power, test_type
"""
alternative = "two-sided" if two_tailed else "larger"
if p1 is not None and p2 is not None:
effect_size = abs(proportion_effectsize(p1, p2))
test_type = "proportion-z"
if effect_size is None:
raise ValueError("Provide effect_size or both p1 and p2.")
if test_type == "two-sample-t":
analysis = TTestIndPower()
n = analysis.solve_power(effect_size=effect_size, alpha=alpha, power=power,
ratio=ratio, alternative=alternative)
elif test_type == "paired-t":
analysis = TTestPower()
n = analysis.solve_power(effect_size=effect_size, alpha=alpha, power=power,
alternative=alternative)
ratio = 1.0
elif test_type == "one-sample-t":
analysis = TTestPower()
n = analysis.solve_power(effect_size=effect_size, alpha=alpha, power=power,
alternative=alternative)
ratio = 1.0
elif test_type == "anova":
# effect_size here is Cohen's f
analysis = FTestAnovaPower()
n = analysis.solve_power(effect_size=effect_size, alpha=alpha, power=power,
k_groups=k_groups)
elif test_type in ("proportion-z", "proportion-chi2"):
analysis = NormalIndPower()
n = analysis.solve_power(effect_size=effect_size, alpha=alpha, power=power,
ratio=ratio, alternative=alternative)
else:
raise ValueError(f"Unknown test_type: {test_type}")
n_ceil = math.ceil(n)
total_n = math.ceil(n_ceil * (1 + ratio)) if test_type not in ("paired-t", "one-sample-t") else n_ceil
result = {
"n_per_group": n_ceil,
"total_n": total_n,
"effect_size": round(effect_size, 4),
"alpha": alpha,
"power": power,
"test_type": test_type,
"ratio": ratio,
}
if verbose:
print(f"Sample size calculation ({test_type})")
print(f" Effect size : {effect_size:.3f}")
print(f" Alpha : {alpha}")
print(f" Power : {power}")
print(f" N per group : {n_ceil}")
print(f" Total N : {total_n}")
if p1 is not None:
print(f" Rates : p1={p1}, p2={p2}")
print()
return result
def mdes(
n_per_group: int,
alpha: float = 0.05,
power: float = 0.80,
test_type: str = "two-sample-t",
) -> float:
"""
Minimum Detectable Effect Size given a fixed sample size.
"""
analysis = TTestIndPower() if test_type == "two-sample-t" else TTestPower()
es = analysis.solve_power(nobs1=n_per_group, alpha=alpha, power=power,
ratio=1.0, alternative="two-sided")
return round(es, 4)
def inflate_for_attrition(n: int, expected_attrition_rate: float = 0.15) -> int:
"""
Inflate sample size to account for expected dropout / missing data.
Parameters
----------
n : int
Required complete-case sample size.
expected_attrition_rate : float
Fraction of participants expected to drop out (0–1).
"""
if not 0 <= expected_attrition_rate < 1:
raise ValueError("attrition_rate must be in [0, 1).")
return math.ceil(n / (1 - expected_attrition_rate))
# ─────────────────────────────────────────────
# 2. Randomization Strategies
# ─────────────────────────────────────────────
def simple_randomize(
n: int,
n_groups: int = 2,
group_labels: Optional[List[str]] = None,
seed: Optional[int] = None,
) -> pd.DataFrame:
"""Simple (unrestricted) randomization."""
rng = np.random.default_rng(seed)
if group_labels is None:
group_labels = [f"Group_{i}" for i in range(n_groups)]
assignments = rng.choice(group_labels, size=n)
df = pd.DataFrame({"participant_id": range(1, n + 1), "assignment": assignments})
return df
def block_randomize(
n: int,
block_size: int = 4,
n_groups: int = 2,
group_labels: Optional[List[str]] = None,
seed: Optional[int] = None,
) -> pd.DataFrame:
"""
Permuted block randomization.
Ensures balance within each block. Block size must be a multiple of n_groups.
Parameters
----------
n : int
Total number of participants to randomize.
block_size : int
Size of each block (must be divisible by n_groups).
n_groups : int
Number of treatment arms.
group_labels : list, optional
seed : int, optional
Returns
-------
DataFrame with participant_id and assignment columns.
"""
if block_size % n_groups != 0:
raise ValueError(f"block_size ({block_size}) must be divisible by n_groups ({n_groups}).")
rng = np.random.default_rng(seed)
if group_labels is None:
group_labels = [f"Group_{i}" for i in range(n_groups)]
per_arm = block_size // n_groups
block_template = group_labels * per_arm
assignments = []
while len(assignments) < n:
block = block_template.copy()
rng.shuffle(block)
assignments.extend(block)
assignments = assignments[:n]
df = pd.DataFrame({"participant_id": range(1, n + 1), "assignment": assignments})
return df
def stratified_randomize(
participants: pd.DataFrame,
stratify_cols: List[str],
block_size: int = 4,
n_groups: int = 2,
group_labels: Optional[List[str]] = None,
seed: Optional[int] = None,
) -> pd.DataFrame:
"""
Stratified block randomization.
Randomizes participants within strata defined by stratify_cols, using
permuted blocks within each stratum to maintain balance.
Parameters
----------
participants : DataFrame
Must contain an 'id' column and the stratify_cols.
stratify_cols : list of str
Columns to stratify on (e.g. ['site', 'sex', 'age_group']).
block_size : int
n_groups : int
group_labels : list, optional
seed : int, optional
Returns
-------
DataFrame with original columns plus 'assignment'.
"""
if group_labels is None:
group_labels = [f"Group_{i}" for i in range(n_groups)]
result_frames = []
# Create stratum key
df = participants.copy().reset_index(drop=True)
df["_stratum"] = df[stratify_cols].astype(str).agg("|".join, axis=1)
for stratum, group_df in df.groupby("_stratum"):
n_stratum = len(group_df)
assigned = block_randomize(
n=n_stratum,
block_size=block_size,
n_groups=n_groups,
group_labels=group_labels,
seed=abs(hash(stratum)) % (2**31) if seed is None else seed + abs(hash(stratum)) % 1000,
)
group_df = group_df.reset_index(drop=True)
group_df["assignment"] = assigned["assignment"].values
result_frames.append(group_df)
result = pd.concat(result_frames).drop(columns=["_stratum"]).sort_index()
return result
def cluster_randomize(
clusters: List[str],
n_groups: int = 2,
group_labels: Optional[List[str]] = None,
seed: Optional[int] = None,
) -> pd.DataFrame:
"""
Cluster randomization — randomize entire clusters to arms.
Parameters
----------
clusters : list of str
Cluster identifiers (e.g. school names, clinic IDs).
n_groups : int
group_labels : list, optional
seed : int, optional
"""
rng = np.random.default_rng(seed)
if group_labels is None:
group_labels = [f"Group_{i}" for i in range(n_groups)]
shuffled_clusters = list(clusters)
rng.shuffle(shuffled_clusters)
assignments = []
for i, cluster in enumerate(shuffled_clusters):
assignments.append({"cluster": cluster, "assignment": group_labels[i % n_groups]})
return pd.DataFrame(assignments)
# ─────────────────────────────────────────────
# 3. AB Test Duration Estimation
# ─────────────────────────────────────────────
def ab_test_duration(
baseline_rate: float,
minimum_detectable_effect: float,
daily_traffic: int,
allocation_fraction: float = 1.0,
alpha: float = 0.05,
power: float = 0.80,
n_variants: int = 2,
) -> Dict[str, float]:
"""
Estimate how many days an A/B test should run.
Parameters
----------
baseline_rate : float
Baseline conversion / success rate (0–1).
minimum_detectable_effect : float
Relative change to detect (e.g. 0.05 = 5% lift).
daily_traffic : int
Total eligible users per day.
allocation_fraction : float
Fraction of traffic included in the test.
alpha : float
power : float
n_variants : int
Number of arms including control.
Returns
-------
dict with required_n, daily_n_in_test, days_needed
"""
p1 = baseline_rate
p2 = baseline_rate * (1 + minimum_detectable_effect)
p2 = min(p2, 0.9999)
size_result = calculate_sample_size(
p1=p1, p2=p2, alpha=alpha / (n_variants - 1), # Bonferroni for multiple arms
power=power, verbose=False
)
required_n = size_result["n_per_group"] * n_variants
daily_n = int(daily_traffic * allocation_fraction / n_variants) * n_variants
days_needed = math.ceil(required_n / daily_n)
return {
"required_n_total": required_n,
"daily_n_in_test": daily_n,
"days_needed": days_needed,
"baseline_rate": p1,
"target_rate": round(p2, 4),
"alpha_adjusted": round(alpha / (n_variants - 1), 4),
}
# ─────────────────────────────────────────────
# 4. Pre-registration Helper
# ─────────────────────────────────────────────
def generate_preregistration_document(
study_title: str,
hypotheses: List[str],
primary_outcome: str,
sample_size_justification: str,
analysis_plan: str,
randomization_method: str,
blinding: str,
output_path: str = "preregistration.md",
) -> str:
"""
Generate a structured pre-registration markdown document.
Upload manually to OSF (https://osf.io) or via the OSF Python client.
"""
lines = [
f"# Pre-registration: {study_title}",
"",
f"**Date:** {pd.Timestamp.now().strftime('%Y-%m-%d')}",
"",
"## Hypotheses",
"",
]
for i, h in enumerate(hypotheses, 1):
lines.append(f"{i}. {h}")
lines += [
"",
"## Primary Outcome",
"",
primary_outcome,
"",
"## Sample Size Justification",
"",
sample_size_justification,
"",
"## Randomization",
"",
randomization_method,
"",
"## Blinding",
"",
blinding,
"",
"## Analysis Plan",
"",
analysis_plan,
"",
"---",
"_This document was auto-generated and should be reviewed before submission._",
]
content = "\n".join(lines)
with open(output_path, "w", encoding="utf-8") as fh:
fh.write(content)
print(f"Pre-registration document written to: {output_path}")
return content
Example 1 — Two-Arm RCT (Clinical Trial)
import pandas as pd
from experimental_design import (
calculate_sample_size,
inflate_for_attrition,
block_randomize,
generate_preregistration_document,
)
# ── Step 1: Power calculation ──────────────────────────────────────────────
# Intervention expected to improve recovery rate from 40% to 55%
result = calculate_sample_size(p1=0.40, p2=0.55, alpha=0.05, power=0.80)
# n_per_group: 128, total_n: 256
# Account for expected 15% dropout
n_inflated = inflate_for_attrition(result["n_per_group"], expected_attrition_rate=0.15)
print(f"Enrol {n_inflated} per arm ({n_inflated * 2} total) to account for 15% attrition.")
# ── Step 2: Block randomization (block size 4, 2 arms) ─────────────────────
allocation = block_randomize(
n=n_inflated * 2,
block_size=4,
n_groups=2,
group_labels=["Control", "Intervention"],
seed=2024,
)
print(allocation.head(8))
print(allocation["assignment"].value_counts())
# ── Step 3: Pre-registration document ─────────────────────────────────────
generate_preregistration_document(
study_title="Effect of Intervention X on Recovery Rate",
hypotheses=[
"Intervention X will increase 30-day recovery rate compared to control.",
"No difference in serious adverse events between arms.",
],
primary_outcome="30-day binary recovery status (recovered / not recovered).",
sample_size_justification=(
f"N={n_inflated} per arm (inflated from {result['n_per_group']} "
f"for 15% attrition). Based on 40% vs 55% recovery rates, "
f"alpha=0.05, power=0.80."
),
analysis_plan=(
"Primary analysis: chi-square test. Secondary: logistic regression "
"adjusting for age and baseline severity."
),
randomization_method="Permuted block randomization, block size 4, centralised allocation.",
blinding="Outcome assessors blinded; participants and clinicians not blinded.",
output_path="rct_preregistration.md",
)
Example 2 — Multi-Arm Web Experiment with Stratification
import pandas as pd
from experimental_design import (
calculate_sample_size,
stratified_randomize,
ab_test_duration,
)
# ── Sample size for 3-arm experiment (ANOVA approach) ─────────────────────
# Cohen's f = 0.25 (medium effect across 3 groups)
result = calculate_sample_size(
effect_size=0.25,
alpha=0.05,
power=0.80,
test_type="anova",
k_groups=3,
)
print(result)
# ── Stratified randomization by country and user_type ─────────────────────
# Simulate a participant pool
rng_sim = pd.np.random.default_rng(99) if hasattr(pd, 'np') else __import__('numpy').random.default_rng(99)
import numpy as np
rng_sim = np.random.default_rng(99)
n_users = result["total_n"]
participants = pd.DataFrame({
"id": range(1, n_users + 1),
"country": np.random.choice(["US", "UK", "DE"], size=n_users, p=[0.5, 0.3, 0.2]),
"user_type": np.random.choice(["free", "paid"], size=n_users, p=[0.7, 0.3]),
})
allocation = stratified_randomize(
participants=participants,
stratify_cols=["country", "user_type"],
block_size=6,
n_groups=3,
group_labels=["Control", "Variant_A", "Variant_B"],
seed=2024,
)
print(allocation.groupby(["country", "user_type", "assignment"]).size().unstack())
# ── AB test duration estimation ────────────────────────────────────────────
duration = ab_test_duration(
baseline_rate=0.03, # 3% baseline conversion
minimum_detectable_effect=0.20, # detect 20% relative lift (3% → 3.6%)
daily_traffic=5000,
allocation_fraction=0.80, # 80% of traffic in the experiment
alpha=0.05,
power=0.80,
n_variants=3,
)
print(f"Run for at least {duration['days_needed']} days.")
Blinding Protocols
| Trial Type | Who Is Blinded |
|---|---|
| Open-label | Nobody (appropriate when blinding is impractical) |
| Single-blind | Participants only |
| Double-blind | Participants + investigators |
| Triple-blind | Participants + investigators + outcome assessors |
| Cluster-blind | Outcome assessors blinded to cluster allocation |
Pre-registration on OSF
# Install OSF client
pip install osfclient
# Authenticate (set token as env var — never hardcode)
export OSF_TOKEN=<paste-your-osf-token>
# Upload pre-registration document to your OSF project
osf -p <your-project-id> upload rct_preregistration.md osfstorage/prereg/rct_preregistration.md
# Verify upload
osf -p <your-project-id> ls
Checklist Before Data Collection
- Primary outcome and analysis plan registered on OSF / ClinicalTrials.gov / AsPredicted
- Sample size calculation documented with effect size source
- Attrition inflation applied
- Randomization sequence sealed (allocation concealment)
- Blinding procedures in place
- Stopping rules defined (safety monitoring)
- IRB / ethics approval obtained
- Data management plan completed
示例提示词
帮我计算这个AB测试所需的样本量和实验运行天数。
所属插件
安装该插件会一并获得同领域的 54 个 Skill / MCP。
同插件的其他科研 Skill
- 智能体资源节约守则Applies token- and context-saving habits for tool choice, PDF handling, persistent memory, and privacy checks. Use at the start of sessions with many files or PDFs, or multi-session work.
- 英文论文双语精读器把英文论文转成中英对照精读档并生成离线双语阅读器,标注要点并输出读书笔记。精读单篇英文论文、要中英对照阅读时使用。
- 研究方法比较论证Compares quantitative, qualitative, and mixed research methods against what each can and cannot answer to build a methods table. Use when justifying a method choice in a thesis.
- 复杂系统分析方法Analyzes complex systems with power-law fitting, Hurst exponent, fractal dimension, agent-based modeling, and entropy measures. Use for nonlinear or complex-system data.
- 研究环境容器化部署Builds Docker and Singularity containers for reproducible computation and HPC deployment of research environments. Use when packaging an analysis pipeline for reuse or HPC.
- DVC数据版本管理Sets up DVC for dataset and pipeline versioning, remote storage, and experiment comparison in reproducible ML research workflows. Use when tracking large datasets or ML experiment runs.
- 期刊级统计图表绘制Creates publication-quality multi-panel figures with matplotlib, seaborn, or ggplot2 using colorblind-safe palettes and journal export settings. Use when preparing figures for a paper.
- 综述模块拆解规划Breaks a refined literature-review question into 3-5 non-overlapping modules to build a reading map, outline, or search plan. Use when structuring a literature review.
- 学位论文答辩教练模拟学位口试委员提问并提供承认回应证据三段式答辩框架与模拟问答。准备论文口试或研讨会问答环节时使用。
- 管理实验设计架构为管理与商学情境实验规划组间组内设计、对抗平衡与操弄检核,并产出设计书与情境材料模板。做商学实验设计时使用。
- 科研经费申请撰写Structures research grant proposals including NSF/NIH Specific Aims, budget justification, ERC narrative sections, and biosketch formatting. Use when drafting a funding application.
- 人类可读文档撰写规范Guides writing human-facing docs like README, PLAN, survey notes, and reviewer-response drafts with a clear two-tier structure. Use for text a human collaborator will read.
- 知识图谱与SPARQL检索Builds knowledge graphs and runs SPARQL queries over RDF and Wikidata for entity linking in research and library data. Use when modeling research entities as a linked graph.
- LaTeX论文写作与排版Writes, compiles, and submits LaTeX papers covering IMRaD structure, key packages, bibliography management, and arXiv preparation. Use when authoring or debugging a LaTeX manuscript.
- 文献比较矩阵构建器把一批PDF整理成含撤稿查核与APA7书目的文献资料库,并生成横向比较矩阵表。整理综述前要比较文献时使用。
- 跨库学术文献检索Searches OpenAlex, Semantic Scholar, and arXiv for academic literature with deduplication, citation snowballing, and BibTeX export. Use when collecting papers for a review or topic.
- 文献调研语料库维护Finds, verifies, and maintains a literature corpus from arXiv, OpenAlex, Crossref, and related databases with BibTeX entries and per-paper notes. Use when sourcing references for a paper.
- 文献综述研究缺口定位Converts a literature-review synthesis into a concrete, defensible research gap and future-research direction. Use when writing a conclusion or thesis contribution section.
- 文献矛盾结论梳理Analyzes conflicting findings across papers and explains disagreement by sample, method, or context differences for a stronger synthesis. Use when review sources disagree.
- 效应量汇总元分析Pools effect sizes across studies with fixed and random-effects models, heterogeneity tests, forest and funnel plots, and subgroup analysis. Use to synthesize quantitative study results.
- 缺失数据处理Diagnoses missing-data mechanisms such as MCAR, MAR, or MNAR and applies multiple imputation or deletion with sensitivity analysis. Use when a dataset has missing values.
- 混合方法研究设计Designs mixed-methods studies integrating qualitative coding, triangulation, content analysis, and QCA with inter-rater reliability checks. Use when combining qual and quant data.
- 科研机器学习流程Applies machine learning to research questions with scikit-learn pipelines, AutoML, SHAP explainability, and nested cross-validation. Use when building a predictive model for a study.
- 开放数据存储与FAIRDeposits and documents research data via Zenodo or OSF with FAIR assessment, data management plans, and DataCite metadata. Use when publishing or sharing a research dataset.
常见问题
实验设计与样本量规划能做什么?
Designs rigorous experiments covering sample-size calculation, randomization strategy, pre-registration, and A/B test duration estimation. Use when planning an experiment or A/B test。
如何安装实验设计与样本量规划?
在科搭 Codar 左侧点击「插件」,在「组件」页签搜索「实验设计与样本量规划」并安装;在 Claude Code 或 Codex 中可以发送本页的安装提示词,或下载 Skill 包后解压到 ~/.claude/skills 或 ~/.codex/skills。
实验设计与样本量规划需要哪些权限?
该科研 Skill没有声明额外权限;执行具体操作时仍由 Agent 按自身规则请求确认。
实验设计与样本量规划经过官方审核吗?
是。SpeedSkills 收录的每个版本都会记录内容摘要与权限声明,当前版本已通过安全审核。