科研 Skill

实验设计与顺序平衡

experimental-design-methods

Guides within- vs between-subjects design, Latin-square and ABBA counterbalancing, confound control, and preregistration before data collection. Use when planning an experiment’s design.

发布者 Codar安全审核通过v0.1.0更新于 2026-11-18

SKILL.md

Experimental Design Methods for Psychology

Purpose

Good analysis cannot fix a design that confounds the manipulation with time, order, or participant characteristics. This skill covers the design-stage decisions psychology researchers must make before any data collection: choosing within- vs between-subjects structure, counterbalancing order effects, controlling common confounds and demand characteristics, and preregistering the design so later analytic choices cannot be inflated by hindsight. It complements, but does not replace, power-analysis (sample size) and psychopy-experiment (software implementation of a chosen design).

When to Use

  • Planning a new experiment and deciding whether conditions should be manipulated within-subjects (each participant sees all conditions) or between-subjects (each participant sees one condition).
  • A within-subjects design needs an order/sequence for presenting conditions, and order itself must not become a confound.
  • Reviewing a draft method section (yours or someone else's) for unconfounded design, before data collection starts.
  • Preparing a preregistration (OSF, AsPredicted) and needing a checklist of design decisions that must be nailed down in advance.

Does NOT trigger

  • Choosing sample size or statistical power for an already-chosen design — use power-analysis.
  • Implementing the chosen design as runnable stimulus-presentation code — use psychopy-experiment.
  • Deciding how to analyze data already collected — this skill is strictly upstream of data collection.

Core Decisions

1. Within-subjects vs. between-subjects vs. mixed

Within-subjectsBetween-subjects
Statistical powerHigher (each person is their own control; removes between-person variance)Lower for the same N
Order/carryover effectsMust be controlled (counterbalancing, below)Not applicable
Demand characteristicsHigher risk (participants see multiple conditions, may guess the hypothesis)Lower risk
Practice/fatigue confoundRisk increases with number of conditions/trialsNot applicable
Required NSmallerLarger (effect size and desired power still set N; see power-analysis)

Mixed designs combine both (e.g., one within-subjects factor, one between-subjects factor — common in 2×2 designs where one factor cannot ethically or logically be manipulated within the same person, such as a clinical-vs-control group factor crossed with a within-subjects task manipulation).

2. Counterbalancing order effects (within-subjects designs)

import itertools
import numpy as np

def latin_square(conditions: list) -> list:
    """
    Build a balanced Latin square: each condition appears exactly once in
    each ordinal position across the square's rows, controlling for simple
    position/practice effects. Classic construction (Bradley, 1958) for
    the balanced (not just any) Latin square, which also balances
    immediate sequential effects for an even number of conditions.
    """
    n = len(conditions)
    square = []
    for i in range(n):
        row = []
        for j in range(n):
            if j % 2 == 0:
                idx = (j // 2 + i) % n
            else:
                idx = (n - (j + 1) // 2 + i) % n
            row.append(conditions[idx])
        square.append(row)
    return square

conditions = ["A", "B", "C", "D"]
square = latin_square(conditions)
for row in square:
    print(row)
# Assign participant p to row (p % n) of the square as their condition order.
def abba_sequence(conditions: list, n_blocks: int) -> list:
    """
    ABBA counterbalancing for 2-condition within-subjects designs: order is
    A,B,B,A,A,B,B,A,... across blocks, which cancels out any LINEAR practice
    or fatigue trend (but not more complex non-linear trends — check block
    position effects post hoc if a non-linear trend is plausible).
    """
    assert len(conditions) == 2, "ABBA counterbalancing is defined for exactly 2 conditions"
    a, b = conditions
    pattern = [a, b, b, a]
    seq = []
    for block in range(n_blocks):
        seq.extend(pattern if block % 2 == 0 else [b, a, a, b])
    return seq

For more than ~6 conditions, a full Latin square becomes impractical to assign one row per participant with enough participants per row; use random assignment of a subset of balanced orders, or random.Random(seed).sample() over a pre-generated pool of balanced sequences, and report how many participants received each order.

3. Controlling confounds and demand characteristics

  • Randomize or counterbalance anything that could otherwise correlate with the manipulation: stimulus order, experimenter (if more than one ran sessions), time of day, testing room/apparatus.
  • Blind what can be blinded: if feasible, the person running the session and/or scoring the outcome should not know condition assignment (single/double blind); when it cannot be done (e.g., participant must know which condition they are in), say so explicitly as a limitation rather than silently.
  • Manipulation check: include a measure that verifies the manipulation had its intended psychological effect (e.g., a mood manipulation should be followed by a mood rating) — without one, a null result is ambiguous between "no effect" and "manipulation failed."
  • Demand characteristics: a cover story, filler tasks, or a funnel debriefing (asking participants what they thought the study was about, from general to specific) helps detect whether participants guessed the hypothesis and behaved accordingly; report the proportion who guessed it and consider excluding (pre-specified in the preregistration) or statistically controlling for them.

4. Preregistration checklist

Before data collection, commit (on OSF, AsPredicted, or a journal's registered-report template) to:

  1. The hypothesis, stated directionally if applicable.
  2. The design (within/between/mixed; number and nature of conditions).
  3. The counterbalancing/randomization scheme (and how participants are assigned to orders/conditions).
  4. Sample size and the power analysis or sequential stopping rule that justifies it (pair with power-analysis).
  5. Exclusion criteria decided in advance (e.g., attention checks, RT cutoffs) — see reaction-time-analysis for RT-specific exclusion conventions.
  6. The primary outcome measure and the exact planned statistical test.
  7. Any manipulation checks and what will be done if they fail (e.g., exclude participants who failed the check — decided now, not after seeing the data).
# Minimal machine-readable preregistration skeleton — adapt fields to the
# target registry's actual schema; this is a planning aid, not a substitute
# for the registry's own form.
prereg = {
    "hypothesis": "...",
    "design": {"type": "within-subjects", "conditions": ["A", "B"], "counterbalancing": "ABBA"},
    "sample_size": {"n_planned": None, "power_analysis_ref": "power-analysis skill output"},
    "primary_outcome": "...",
    "primary_test": "...",
    "exclusion_criteria": ["..."],
    "manipulation_check": "...",
}

Prerequisites

  • A specific, falsifiable hypothesis and at least a draft list of conditions before applying the counterbalancing schemes above — these tools assume the conditions are already decided, not that they will help choose them.
  • For registering the design, an account on the target registry (OSF, AsPredicted, or a journal's own registered-report portal); this skill produces the planning content, not the registry submission itself.
  • Basic familiarity with the planned statistical test, since the design (within vs. between, number of factors) determines which test is valid — decide the test and the design together, not the design first and the test as an afterthought.

Limitations

  • The Latin square and ABBA schemes above control for linear and simple position order effects; they do not guarantee immunity from differential carryover (where the effect of condition A on a later trial depends on whether it followed B specifically) — a full "digram-balanced" Latin square or treating order as an explicit factor in the analysis is needed if differential carryover is a serious concern.
  • This skill does not cover quasi-experimental or observational designs (e.g., using pre-existing groups) in any depth — those need their own confound/selection-bias reasoning (e.g., propensity matching) and belong with causal-inference-focused skills rather than here.
  • Preregistration locks the planned analysis, but does not prevent or excuse later deviations; any departure from the preregistered plan should be reported explicitly as exploratory, not folded silently into the confirmatory results.

Quality Checks

  • Before finalizing a within-subjects design, confirm that the chosen counterbalancing scheme actually balances order against the specific hypothesis test planned (e.g., balancing position is not enough if the hypothesis concerns an interaction with trial number).
  • Check that the number of participants is divisible (or close to it) by the number of counterbalancing orders used, so no single order is under-represented.
  • Confirm a manipulation check (or an explicit justification for omitting one) exists before data collection — this is far cheaper to add at design time than to discover its absence after a null result.
  • Verify the preregistration's primary outcome and test match what the power analysis was actually computed for; a mismatch here is a common, avoidable inconsistency between the registered plan and the analysis plan.

References

  • Shadish, W. R., Cook, T. D., & Campbell, D. T. (2002). Experimental and Quasi-Experimental Designs for Generalized Causal Inference.
  • Bradley, J. V. (1958). Complete counterbalancing of immediate sequential effects in a Latin square design. Journal of the American Statistical Association, 53(282), 525–528.
  • Orne, M. T. (1962). On the social psychology of the psychological experiment: With particular reference to demand characteristics. American Psychologist, 17(11), 776–783.
  • Nosek, B. A., Ebersole, C. R., DeHaven, A. C., & Mellor, D. T. (2018). The preregistration revolution. PNAS, 115(11), 2600–2606.
  • Open Science Framework preregistration templates: https://www.cos.io/initiatives/prereg

示例提示词

我想为这个被试内实验设计拉丁方序列并写一份预注册方案。

所属插件

心理学面向心理学研究:心理测量与量表修订、结构方程、多层模型、反应时与眼动分析、PsychoPy 实验、功效分析与心理学元分析。

安装该插件会一并获得同领域的 13 个 Skill / MCP。

同插件的其他科研 Skill

常见问题

实验设计与顺序平衡能做什么?

Guides within- vs between-subjects design, Latin-square and ABBA counterbalancing, confound control, and preregistration before data collection. Use when planning an experiment’s design。

如何安装实验设计与顺序平衡?

在科搭 Codar 左侧点击「插件」,在「组件」页签搜索「实验设计与顺序平衡」并安装;在 Claude Code 或 Codex 中可以发送本页的安装提示词,或下载 Skill 包后解压到 ~/.claude/skills 或 ~/.codex/skills。

实验设计与顺序平衡需要哪些权限?

该科研 Skill没有声明额外权限;执行具体操作时仍由 Agent 按自身规则请求确认。

实验设计与顺序平衡经过官方审核吗?

是。SpeedSkills 收录的每个版本都会记录内容摘要与权限声明,当前版本已通过安全审核。