科研 Skill
实验设计与顺序平衡
experimental-design-methods
Guides within- vs between-subjects design, Latin-square and ABBA counterbalancing, confound control, and preregistration before data collection. Use when planning an experiment’s design.
SKILL.md
Experimental Design Methods for Psychology
Purpose
Good analysis cannot fix a design that confounds the manipulation with time,
order, or participant characteristics. This skill covers the design-stage
decisions psychology researchers must make before any data collection:
choosing within- vs between-subjects structure, counterbalancing order
effects, controlling common confounds and demand characteristics, and
preregistering the design so later analytic choices cannot be inflated by
hindsight. It complements, but does not replace, power-analysis (sample
size) and psychopy-experiment (software implementation of a chosen design).
When to Use
- Planning a new experiment and deciding whether conditions should be manipulated within-subjects (each participant sees all conditions) or between-subjects (each participant sees one condition).
- A within-subjects design needs an order/sequence for presenting conditions, and order itself must not become a confound.
- Reviewing a draft method section (yours or someone else's) for unconfounded design, before data collection starts.
- Preparing a preregistration (OSF, AsPredicted) and needing a checklist of design decisions that must be nailed down in advance.
Does NOT trigger
- Choosing sample size or statistical power for an already-chosen design —
use
power-analysis. - Implementing the chosen design as runnable stimulus-presentation code —
use
psychopy-experiment. - Deciding how to analyze data already collected — this skill is strictly upstream of data collection.
Core Decisions
1. Within-subjects vs. between-subjects vs. mixed
| Within-subjects | Between-subjects | |
|---|---|---|
| Statistical power | Higher (each person is their own control; removes between-person variance) | Lower for the same N |
| Order/carryover effects | Must be controlled (counterbalancing, below) | Not applicable |
| Demand characteristics | Higher risk (participants see multiple conditions, may guess the hypothesis) | Lower risk |
| Practice/fatigue confound | Risk increases with number of conditions/trials | Not applicable |
| Required N | Smaller | Larger (effect size and desired power still set N; see power-analysis) |
Mixed designs combine both (e.g., one within-subjects factor, one between-subjects factor — common in 2×2 designs where one factor cannot ethically or logically be manipulated within the same person, such as a clinical-vs-control group factor crossed with a within-subjects task manipulation).
2. Counterbalancing order effects (within-subjects designs)
import itertools
import numpy as np
def latin_square(conditions: list) -> list:
"""
Build a balanced Latin square: each condition appears exactly once in
each ordinal position across the square's rows, controlling for simple
position/practice effects. Classic construction (Bradley, 1958) for
the balanced (not just any) Latin square, which also balances
immediate sequential effects for an even number of conditions.
"""
n = len(conditions)
square = []
for i in range(n):
row = []
for j in range(n):
if j % 2 == 0:
idx = (j // 2 + i) % n
else:
idx = (n - (j + 1) // 2 + i) % n
row.append(conditions[idx])
square.append(row)
return square
conditions = ["A", "B", "C", "D"]
square = latin_square(conditions)
for row in square:
print(row)
# Assign participant p to row (p % n) of the square as their condition order.
def abba_sequence(conditions: list, n_blocks: int) -> list:
"""
ABBA counterbalancing for 2-condition within-subjects designs: order is
A,B,B,A,A,B,B,A,... across blocks, which cancels out any LINEAR practice
or fatigue trend (but not more complex non-linear trends — check block
position effects post hoc if a non-linear trend is plausible).
"""
assert len(conditions) == 2, "ABBA counterbalancing is defined for exactly 2 conditions"
a, b = conditions
pattern = [a, b, b, a]
seq = []
for block in range(n_blocks):
seq.extend(pattern if block % 2 == 0 else [b, a, a, b])
return seq
For more than ~6 conditions, a full Latin square becomes impractical to
assign one row per participant with enough participants per row; use random
assignment of a subset of balanced orders, or random.Random(seed).sample()
over a pre-generated pool of balanced sequences, and report how many
participants received each order.
3. Controlling confounds and demand characteristics
- Randomize or counterbalance anything that could otherwise correlate with the manipulation: stimulus order, experimenter (if more than one ran sessions), time of day, testing room/apparatus.
- Blind what can be blinded: if feasible, the person running the session and/or scoring the outcome should not know condition assignment (single/double blind); when it cannot be done (e.g., participant must know which condition they are in), say so explicitly as a limitation rather than silently.
- Manipulation check: include a measure that verifies the manipulation had its intended psychological effect (e.g., a mood manipulation should be followed by a mood rating) — without one, a null result is ambiguous between "no effect" and "manipulation failed."
- Demand characteristics: a cover story, filler tasks, or a funnel debriefing (asking participants what they thought the study was about, from general to specific) helps detect whether participants guessed the hypothesis and behaved accordingly; report the proportion who guessed it and consider excluding (pre-specified in the preregistration) or statistically controlling for them.
4. Preregistration checklist
Before data collection, commit (on OSF, AsPredicted, or a journal's registered-report template) to:
- The hypothesis, stated directionally if applicable.
- The design (within/between/mixed; number and nature of conditions).
- The counterbalancing/randomization scheme (and how participants are assigned to orders/conditions).
- Sample size and the power analysis or sequential stopping rule that
justifies it (pair with
power-analysis). - Exclusion criteria decided in advance (e.g., attention checks, RT
cutoffs) — see
reaction-time-analysisfor RT-specific exclusion conventions. - The primary outcome measure and the exact planned statistical test.
- Any manipulation checks and what will be done if they fail (e.g., exclude participants who failed the check — decided now, not after seeing the data).
# Minimal machine-readable preregistration skeleton — adapt fields to the
# target registry's actual schema; this is a planning aid, not a substitute
# for the registry's own form.
prereg = {
"hypothesis": "...",
"design": {"type": "within-subjects", "conditions": ["A", "B"], "counterbalancing": "ABBA"},
"sample_size": {"n_planned": None, "power_analysis_ref": "power-analysis skill output"},
"primary_outcome": "...",
"primary_test": "...",
"exclusion_criteria": ["..."],
"manipulation_check": "...",
}
Prerequisites
- A specific, falsifiable hypothesis and at least a draft list of conditions before applying the counterbalancing schemes above — these tools assume the conditions are already decided, not that they will help choose them.
- For registering the design, an account on the target registry (OSF, AsPredicted, or a journal's own registered-report portal); this skill produces the planning content, not the registry submission itself.
- Basic familiarity with the planned statistical test, since the design (within vs. between, number of factors) determines which test is valid — decide the test and the design together, not the design first and the test as an afterthought.
Limitations
- The Latin square and ABBA schemes above control for linear and simple position order effects; they do not guarantee immunity from differential carryover (where the effect of condition A on a later trial depends on whether it followed B specifically) — a full "digram-balanced" Latin square or treating order as an explicit factor in the analysis is needed if differential carryover is a serious concern.
- This skill does not cover quasi-experimental or observational designs (e.g., using pre-existing groups) in any depth — those need their own confound/selection-bias reasoning (e.g., propensity matching) and belong with causal-inference-focused skills rather than here.
- Preregistration locks the planned analysis, but does not prevent or excuse later deviations; any departure from the preregistered plan should be reported explicitly as exploratory, not folded silently into the confirmatory results.
Quality Checks
- Before finalizing a within-subjects design, confirm that the chosen counterbalancing scheme actually balances order against the specific hypothesis test planned (e.g., balancing position is not enough if the hypothesis concerns an interaction with trial number).
- Check that the number of participants is divisible (or close to it) by the number of counterbalancing orders used, so no single order is under-represented.
- Confirm a manipulation check (or an explicit justification for omitting one) exists before data collection — this is far cheaper to add at design time than to discover its absence after a null result.
- Verify the preregistration's primary outcome and test match what the power analysis was actually computed for; a mismatch here is a common, avoidable inconsistency between the registered plan and the analysis plan.
References
- Shadish, W. R., Cook, T. D., & Campbell, D. T. (2002). Experimental and Quasi-Experimental Designs for Generalized Causal Inference.
- Bradley, J. V. (1958). Complete counterbalancing of immediate sequential effects in a Latin square design. Journal of the American Statistical Association, 53(282), 525–528.
- Orne, M. T. (1962). On the social psychology of the psychological experiment: With particular reference to demand characteristics. American Psychologist, 17(11), 776–783.
- Nosek, B. A., Ebersole, C. R., DeHaven, A. C., & Mellor, D. T. (2018). The preregistration revolution. PNAS, 115(11), 2600–2606.
- Open Science Framework preregistration templates: https://www.cos.io/initiatives/prereg
示例提示词
我想为这个被试内实验设计拉丁方序列并写一份预注册方案。
所属插件
安装该插件会一并获得同领域的 13 个 Skill / MCP。
同插件的其他科研 Skill
- 结构方程与测量模型拟合Fits CFA, SEM, EFA, and IRT models with semopy and factor_analyzer, reporting fit indices, omega reliability, and invariance. Use for latent-variable, factor-loading, or invariance requests.
- 临床量表评分解读Scores PHQ-9, GAD-7, PCL-5, BDI-II, and SWLS, computing severity, prorated scores, and the Reliable Change Index. Use when batch-scoring clinical scales or testing pre-post change.
- 眼动数据分析Processes raw eye-tracking data: IVT fixation detection, AOI dwell time, scanpath similarity, heatmaps, pupillometry baseline correction. Use for gaze or pupil data from eye trackers.
- 评分者间信度分析Computes inter-rater/inter-coder reliability with Cohen’s/weighted/Fleiss’ kappa, Krippendorff’s alpha, ICC, and Gwet’s AC1. Use when raters coded the same cases and agreement must be reported.
- 心理学元分析Pools effect sizes across studies with Cohen’s d, Hedges’ g, random-effects pooling, forest/funnel plots, and publication-bias tests. Use when synthesizing results from multiple studies.
- 多层线性混合效应模型Fits multilevel/mixed-effects models with pymer4 and lme4: ICC, random slopes, likelihood-ratio comparison, within-person centering for ESM data. Use for nested or repeated-measures designs.
- 心理学研究功效分析Computes power, sample size, or minimum detectable effect for t-tests, ANOVA, correlation, regression, and simulation-based designs including mediation. Use when planning a study’s sample size.
- 心理测量学分析Runs CTT item stats, Cronbach’s alpha, EFA, CFA, 2PL IRT, measurement invariance, and Mantel-Haenszel DIF detection in Python. Use when validating a scale’s reliability, structure, or item fairness.
- PsychoPy行为实验设计Builds PsychoPy experiments: monitor calibration, stimulus presentation, response timing, BIDS events.tsv output, Pavlovia deployment checklist. Use when implementing a stimulus-response task.
- 反应时数据分析Cleans RT data and fits ex-Gaussian distributions, EZ-diffusion parameters, and Bayesian drift-diffusion models with PyMC. Use for RT outliers, distribution shape, or decision-model parameters.
- 单被试实验设计分析Analyzes single-case ABAB, multiple-baseline, and alternating-treatment designs with visual analysis, Tau-U, PND, and randomization tests. Use for studies with few participants observed across phases.
- 问卷量表翻译审查Reviews OB, management, and psychology survey scales for translation equivalence and adaptation validity, producing a Chinese dashboard report. Use for pre-launch questionnaire quality checks.
常见问题
实验设计与顺序平衡能做什么?
Guides within- vs between-subjects design, Latin-square and ABBA counterbalancing, confound control, and preregistration before data collection. Use when planning an experiment’s design。
如何安装实验设计与顺序平衡?
在科搭 Codar 左侧点击「插件」,在「组件」页签搜索「实验设计与顺序平衡」并安装;在 Claude Code 或 Codex 中可以发送本页的安装提示词,或下载 Skill 包后解压到 ~/.claude/skills 或 ~/.codex/skills。
实验设计与顺序平衡需要哪些权限?
该科研 Skill没有声明额外权限;执行具体操作时仍由 Agent 按自身规则请求确认。
实验设计与顺序平衡经过官方审核吗?
是。SpeedSkills 收录的每个版本都会记录内容摘要与权限声明,当前版本已通过安全审核。