科研 Skill

动物行为姿态追踪

animal-behavior-tracking

Tracks animal pose and behavior from video with DeepLabCut or SLEAP, and extracts kinematic and behavioral event features. Use for markerless tracking or scoring behavior.

发布者 Codar安全审核通过v0.1.0更新于 2026-11-18

SKILL.md

Animal behavior tracking

Markerless pose estimation (DeepLabCut, SLEAP) turns raw behavior video into labeled body-part trajectories, which then need cleaning, kinematic feature extraction, and often event/state classification before they are useful for correlating with neural data. This skill covers the pipeline from raw video to analysis-ready behavioral features.

When to use this skill

Use it when the user has video of a freely moving or head-fixed animal and needs body-part or whole-animal tracking (DeepLabCut, SLEAP, LEAP, or similar), wants to label training frames, is debugging bad tracking (jumping keypoints, low confidence, identity swaps in multi-animal video), needs kinematic features (speed, distance traveled, joint angles) from existing tracking output, or is building a behavior classifier (freezing, rearing, grooming, social contact) from either tracking output or raw video (e.g. with SimBA, JAABA, or a custom classifier).

1. Choose a tracking tool

ToolBest forNotes
DeepLabCut (DLC)Single or multi-animal, most widely adopted, large community model zoopip install deeplabcut; GUI or Python API; supports multi-animal (maDLC) with identity tracking
SLEAPMulti-animal, social behavior, fast inference, active developmentpip install sleap; strong multi-animal identity tracking and GUI labeling
LEAP / Anipose3D multi-camera poseAnipose wraps DLC/SLEAP for calibrated multi-camera 3D reconstruction

Both DLC and SLEAP are supervised: they need a set of manually labeled frames to train a model for the specific camera angle, lighting and animal/apparatus in use. A model trained on one rig's lighting and camera angle commonly fails on another rig — budget for new labeling, or at least fine-tuning, per rig/condition rather than assuming a published model transfers directly.

2. Label training frames

  • Extract a diverse set of frames (DLC's extract_frames, SLEAP's suggestion-based frame selection) spanning the full range of poses, occlusions, lighting, and (for multi-animal) proximity/overlap conditions expected in the real data — not just easy, well-separated frames.
  • 100-200 labeled frames is a reasonable starting point for a single animal in a simple arena; complex poses, multiple animals, or variable lighting need substantially more.
  • Define keypoints (body parts) that are visible from the camera angle in most frames; a keypoint that is occluded in half the video will have low-confidence, noisy tracking throughout.
  • For multi-animal tracking, label identity consistently across frames (which animal is which) since identity-swap errors are the dominant failure mode in social/group video.

3. Train and evaluate

# DeepLabCut: minimal project workflow
import deeplabcut

config_path = deeplabcut.create_new_project(
    "MyBehaviorProject", "experimenter", ["/path/to/video1.mp4"],
    working_directory="/path/to/project", copy_videos=False,
)
deeplabcut.extract_frames(config_path, mode="automatic", algo="kmeans")
# ... label frames in the GUI: deeplabcut.label_frames(config_path) ...
deeplabcut.create_training_dataset(config_path)
deeplabcut.train_network(config_path, maxiters=100000)
deeplabcut.evaluate_network(config_path, plotting=True)
deeplabcut.analyze_videos(config_path, ["/path/to/new_video.mp4"])
# SLEAP: minimal CLI workflow (after labeling in the SLEAP GUI)
# sleap-train baseline_model.json labels.slp
# sleap-track video.mp4 -m models/baseline_model
import sleap
labels = sleap.load_file("predictions.slp")

Evaluate on a held-out labeled test set before trusting predictions on new video: report pixel error (train vs. test) and inspect the worst-error frames directly, since mean pixel error alone hides systematic failures (e.g. a keypoint that is always correct except during rearing).

4. Clean the raw tracking output

Raw pose estimates from any tracker contain:

  • Low-confidence/missing frames: every tool outputs a per-keypoint confidence (DLC: likelihood; SLEAP: score). Threshold on it (commonly ~0.6-0.9, tool- and dataset-dependent) and treat below-threshold points as missing rather than using them uncritically.
  • Jumps: a keypoint teleporting between frames, usually from low-confidence misdetections. Filter with a median filter, a velocity/jump threshold, or a Kalman smoother; DLC's companion package dlc2kinematics and community tools provide some of this, or implement a simple velocity-threshold interpolation.
  • Identity swaps (multi-animal): check identity continuity frame to frame; both DLC (via tracklets) and SLEAP provide tools to stitch and correct identity assignments after tracking, which generally still need manual review on a sample of swap events.
  • Missing-data interpolation: short gaps (a few frames) can be linearly interpolated; long gaps (occlusion behind an object) should be left as missing rather than interpolated across, since interpolation invents kinematics that were not observed.

5. Extract kinematic and behavioral features

From cleaned body-part trajectories (in pixels, then converted to real-world units via a calibration — a ruler or checkerboard of known size in the frame, or full camera calibration for Anipose 3D data):

  • Locomotion: centroid speed, distance traveled, time in zones (for open-field/place-preference assays), turning angle.
  • Posture/kinematics: joint angles (e.g. hip-knee-ankle for gait), body-part distances (e.g. nose-to-tail for stretching/rearing), head direction.
  • Events/states: define behavior bouts (freezing, grooming, rearing, social approach) either by hand-crafted thresholds on kinematic features (e.g. freezing = centroid speed below a threshold for at least N consecutive frames) or with a supervised classifier trained on hand-labeled bouts (SimBA, JAABA, B-SOiD for unsupervised discovery of behavioral syllables).
  • Always report the frame rate and any temporal smoothing applied before computing speed/acceleration — pose-estimation noise is amplified by differentiation, and under-smoothed speed traces are a common source of spurious "fast" events.

6. Align with neural data

To correlate behavior with simultaneously recorded neural signals, the camera frame times must be on the same clock as the neural recording: use a hardware frame-timestamp signal (camera strobe/frame-sync pulse recorded on an aux channel) rather than assuming a nominal frame rate, following the same logic as neuro-sync-forensics. A camera's actual frame rate commonly drifts from its nominal rate (dropped frames, USB bandwidth limits), which silently misaligns behavior and neural events if not checked.

7. Quality checks

  • Spot-check tracking on representative video clips by overlaying predicted keypoints, including clips the model was not trained or validated on.
  • Report the fraction of frames below the confidence threshold per keypoint; a keypoint that is frequently low-confidence needs either a camera-angle change or more targeted labeling, not just a lower threshold.
  • For any derived behavior classifier, report precision/recall against a held-out set of hand-scored bouts, not just visual agreement.

Output

A short tracking report: tool and model (or model zoo source), number of training frames and keypoints, train/test pixel error, confidence threshold used and fraction of frames excluded per keypoint, cleaning steps applied (jump filtering, interpolation, identity-swap correction), pixel-to-real-world calibration, and the kinematic/behavioral features extracted with their definitions and thresholds.

Prerequisites

  • Video with adequate resolution and frame rate for the behavior of interest (fast movements like whisking or paw kinematics need higher frame rates than general locomotion), and consistent lighting/camera position across sessions if comparing across days.
  • GPU strongly recommended for training (DLC/SLEAP both support CPU inference but training on CPU is impractically slow for most datasets).
  • A known physical scale in the frame (ruler, checkerboard, or known arena dimensions) to convert pixel coordinates to real-world units.

Limitations

  • Pose-estimation models are rig- and condition-specific; accuracy on video that differs from the training distribution (new camera angle, lighting, animal strain/coat color, occluding objects) should not be assumed and must be checked against held-out labeled frames from the new condition.
  • This skill produces kinematic and behavioral-event features; it does not itself provide a validated ethogram or behavioral taxonomy — classifier outputs (freezing, rearing, etc.) should be validated against independent human scoring before being treated as ground truth.
  • 3D reconstruction from multiple cameras (Anipose or similar) requires a proper multi-camera calibration; skipping calibration and assuming approximate geometry produces systematic distance and angle errors.

示例提示词

帮我用DeepLabCut追踪这段开野实验视频里小鼠的姿态并提取运动特征。

所属插件

神经科学面向神经科学研究:EEG/MEG、fMRI、BIDS、spike sorting、连接组、脑机接口解码评估、神经刺激安全与公开神经数据检索。

安装该插件会一并获得同领域的 45 个 Skill / MCP。

同插件的其他科研 Skill

常见问题

动物行为姿态追踪能做什么?

Tracks animal pose and behavior from video with DeepLabCut or SLEAP, and extracts kinematic and behavioral event features. Use for markerless tracking or scoring behavior。

如何安装动物行为姿态追踪?

在科搭 Codar 左侧点击「插件」,在「组件」页签搜索「动物行为姿态追踪」并安装;在 Claude Code 或 Codex 中可以发送本页的安装提示词,或下载 Skill 包后解压到 ~/.claude/skills 或 ~/.codex/skills。

动物行为姿态追踪需要哪些权限?

该科研 Skill没有声明额外权限;执行具体操作时仍由 Agent 按自身规则请求确认。

动物行为姿态追踪经过官方审核吗?

是。SpeedSkills 收录的每个版本都会记录内容摘要与权限声明,当前版本已通过安全审核。