科研 Skill
计算材料科学可复现性管理
reproducibility-workflow
Covers provenance tracking, data versioning, and environment capture for reproducible computational materials campaigns: DFT dataset. Use when documenting a simulation campaign for reuse.
SKILL.md
Reproducibility Workflow
Description
This skill covers the systematic provenance tracking, data versioning, and environment capture required to make computational materials science campaigns reproducible: DFT dataset provenance, MLP training experiment tracking, active learning iteration versioning, environment snapshots, model registry management, and data integrity verification. Invoke this skill when designing a new simulation or ML campaign from scratch, when auditing an inherited dataset or model, or when preparing data and results for publication or long-term archiving.
Domain Context
Reproducibility in computational materials science and scientific ML has two distinct failure modes that are easy to confuse. The first is bit-exact reproducibility: given identical inputs, code, and hardware, the same numbers appear. The second is scientific reproducibility: an independent researcher using the same protocol and nominally identical inputs arrives at the same scientific conclusions. The first is often achievable for DFT and classical MD; it becomes harder for GPU-accelerated ML training where floating-point nondeterminism is inherent. The second requires that the protocol itself — inputs, settings, preprocessing steps, and analysis code — is fully recorded.
In computational materials science, the most common reproducibility failures are not deliberate falsification but silent data corruption: a POTCAR file quietly updated mid-campaign, a preprocessing script run without recording its parameters, a train/test split generated with no fixed random seed, a model checkpoint overwritten by a re-run. Each of these produces results that look correct locally but cannot be reproduced six months later, or produce subtly different results when the dataset is extended.
The provenance chain in a typical MLP campaign spans at least four levels:
- DFT calculations: Code version, pseudopotentials (checksums), functional, ENCUT/ecutwfc, k-mesh density, smearing, convergence thresholds, cell, PBC, MAGMOM. A change at this level invalidates all downstream training.
- Dataset assembly: Which raw calculations were included, which were filtered (and why), the extxyz conversion script, the train/val/test split seed, and the resulting dataset file checksums.
- Model training: Architecture hyperparameters, loss weights, random seed, training data file checksum, optimizer settings, number of epochs, the exact training command, and the validation metrics at the best checkpoint.
- Deployment and production: Model file checksum, LAMMPS version and plugin hash, simulation input script, initial structure, random seed for velocity initialization, ensemble and thermostat settings.
FAIR data principles (Findable, Accessible, Interoperable, Reusable) are the upstream standard for materials data. Reproducibility is a prerequisite for FAIR reusability: data that cannot be reproduced cannot be reused. DFT dataset provenance metadata (functional, pseudopotentials, k-mesh) maps directly to the "Reusable" and "Interoperable" FAIR criteria and must be machine-readable, not buried in a paper's supplementary PDF.
The tension between cost and completeness. Full provenance tracking has overhead: every DFT run writes a sidecar JSON; every preprocessing step logs its inputs and outputs; every training run is tracked in MLflow. This overhead is negligible compared to DFT wall time and MLP training GPU-hours, but it requires upfront infrastructure setup. The alternative — tracking provenance manually or reconstructing it post-hoc — is expensive and error-prone at campaign scale (hundreds of DFT calculations, multiple AL iterations, dozens of training runs).
When to Use This Skill
- Setting up the infrastructure for a new MLP training campaign before any DFT calculations are submitted.
- When a dataset is being shared with collaborators or submitted to a data repository (NOMAD, Materials Cloud, Zenodo).
- When preparing the methods section of a paper that reports MLP training results — the provenance metadata is the source of truth for what was actually computed.
- When inheriting a dataset or model from a prior project and needing to audit whether it is internally consistent and correctly annotated.
- When an active learning campaign runs for multiple iterations and the connection between model version N and the DFT batch that produced it must be unambiguous.
- When debugging an unexpected discrepancy between two runs that should have been identical.
When Not to Use This Skill
- For purely exploratory single-run calculations where no downstream use is planned — full provenance overhead is not justified for a one-off convergence test that will be discarded.
- As a substitute for proper experimental design — tracking provenance of a poorly designed dataset does not fix the dataset.
- When the overhead of Git-LFS or DVC exceeds the benefit — for very small datasets (< 100 structures) stored in a private directory, a well-organized folder with checksum files and a README may be sufficient.
- For real-time or latency-sensitive workflows — provenance metadata writes add I/O overhead; they are appropriate for batch HPC workflows, not for low-latency inference services.
Core Concepts
- Git commit hash: A 40-character SHA-1 identifier that uniquely and immutably identifies the state of the codebase at a given point. Every production run script, analysis notebook, and LAMMPS input file should be version-controlled; the Git commit hash at run time should be recorded in the run's metadata.
- DVC (Data Version Control): A Git-compatible version control system for large binary files (datasets, model weights) that Git was not designed to handle. DVC stores a small
.dvcpointer file in Git while the actual data resides in a remote storage (S3, GCS, SSH). DVC enables dataset versioning with the same semantics as code versioning. - MLflow: An experiment tracking platform. MLflow records hyperparameters (
mlflow.log_param), metrics (mlflow.log_metric), artifacts (model files, figures, configs), and environment information for each training run. Provides a UI for comparing runs and a model registry for versioning production models. - SHA-256 file hash: A 256-bit content hash that uniquely identifies a file's contents regardless of filename, path, or metadata. Used to detect silent file corruption, verify POTCAR identity, and confirm that the model file deployed in LAMMPS is the same one reported in the training log.
hashlib.sha256(open(f,"rb").read()).hexdigest(). - Random seed: An integer used to initialize a pseudorandom number generator. Fixing the seed in Python (
random.seed,np.random.seed,torch.manual_seed), in LAMMPS velocity initialization (velocity ... create ... seed), and in train/test splitting ensures that the same sequence of random decisions is made in every run. Without a fixed seed, two runs on the same data can produce different train/test splits, different model initializations, and different MD initial conditions. - Environment snapshot: A machine-readable record of all software versions used in a computation: Python package versions (
conda env export,pip freeze), compiler versions (gcc --version,mpicc --version), CUDA version, LAMMPS version, and any HPC modules loaded. Required for reproducing results on a different machine or after a system upgrade. - Immutable dataset: A dataset file (extxyz, HDF5, or directory) that is never modified in place. Once a dataset version is created, it receives a checksum and is treated as read-only. Any modification (filtering, adding structures) creates a new version with a new checksum. The link between dataset version and model version must be explicit.
- Provenance sidecar: A small JSON file written alongside each computational artifact (DFT output directory, model checkpoint, extxyz file) recording the inputs, settings, code version, and checksums that produced it. Sidecars are co-located with their artifacts and survive directory moves better than a centralized database.
- Model registry: A catalog of trained model versions with associated metadata: training data checksum, validation metrics, deployment status (staging / production / archived), and creation timestamp. MLflow provides a built-in registry; a simple JSON file is an acceptable minimum.
- Idempotent pipeline: A data processing pipeline where running the same step twice with the same inputs produces exactly the same output. Idempotent pipelines are a prerequisite for reproducibility. Steps that depend on system time, network calls, or uncontrolled random state are not idempotent.
Key Workflows
Full steps for each part are in references/key-workflows/; open the one you need:
- Workflow 1: Git + DVC repository setup for a simulation campaign —
references/key-workflows/01-workflow-1-git-dvc-repository-setup-for-a-simula.md: Initialize a version-controlled repository for a new MLP training campaign. The setup below assumes a standard campaign structure: raw DFT calculations → assembled dataset → trained model → validation results. - Workflow 2: DFT provenance capture per calculation —
references/key-workflows/02-workflow-2-dft-provenance-capture-per-calculatio.md: Extend the per-calculation provenance sidecar beyond whatDatasetManifesttracks (seedft-dataset-generationWorkflow 6), capturing everything needed to re-run or audit a single DFT calculation. - Workflow 3: MLP training experiment tracking with MLflow —
references/key-workflows/03-workflow-3-mlp-training-experiment-tracking-with.md: Track every MACE training run as an MLflow experiment. This enables comparison across runs, model registry management, and reproducibility of the best model. - Workflow 4: Environment capture and snapshot —
references/key-workflows/04-workflow-4-environment-capture-and-snapshot.md: A computation cannot be reproduced without knowing the exact software environment. Record Python packages, compilers, CUDA, MPI, and HPC modules at run time. - Workflow 5: Dataset integrity verification and hash registry —
references/key-workflows/05-workflow-5-dataset-integrity-verification-and-ha.md: Build and verify a hash registry for all datasets, model files, and critical input files. Detect silent corruption or accidental modification. - Workflow 6: Active learning campaign provenance chain —
references/key-workflows/06-workflow-6-active-learning-campaign-provenance-c.md: An active learning campaign spans multiple iterations, each adding DFT structures and retraining the model. Maintaining an unambiguous connection between model version N and the DFT batch that produced it is the central provenance challeng…
Common Failure Modes
| Failure | Symptom | Likely Cause | Diagnostic | Fix |
|---|---|---|---|---|
| Overwritten dataset | Training from "the latest version" of train.xyz produces different model each time; no record of which structures were in the dataset at a given model checkpoint | extxyz files written in-place without versioning; DVC not used for large dataset files; write(output_file, frames) overwrites prior iteration | Check git log -- datasets/train.xyz; if there is only one commit, the file was never tracked with history; run dvc diff | Use DVC to track all extxyz dataset files; name files with iteration suffix (train_iter002.xyz) rather than generic names; never use write to overwrite a file that is referenced by an existing model checkpoint |
| Untracked preprocessing | Results not reproducible from raw DFT outputs despite having all raw files; a filtering step (FPS, convergence check, unit conversion) was applied but not recorded | Preprocessing script run interactively in a notebook or terminal without being committed to Git; script modified between runs; --seed not set for FPS | Check git log scripts/; if the preprocessing script has uncommitted changes or is absent, the exact transformation is unknown; verify extxyz dataset frame count matches what the script would produce from the raw inputs | Commit all preprocessing scripts before running them; record the script's Git commit hash and all CLI arguments in the dataset sidecar; use DVC pipeline stages (dvc.yaml) to declare preprocessing as a tracked step |
| Mixed code versions across AL iterations | Training loss increases when adding DFT data from a later iteration; systematic energy offset between iteration 0 and iteration 3 structures | VASP was upgraded mid-campaign (different default POTCAR behavior or FFT grid precision); QE pseudopotential downloaded with a different version; Python environment updated mid-campaign | Compare VASP version string from OUTCAR across all iterations; compare POTCAR SHA-256 checksums across iteration directories; run compare_environments on snapshots from each iteration | Record code version and POTCAR checksums at every DFT batch (Workflow 2); enforce via DatasetManifest.register_settings hash check; if version mismatch found, recompute the affected structures |
| Missing pseudopotential hashes | POTCAR or UPF file silently updated; energy reference surface shifts by 0.01–0.1 eV/atom; apparent improvement in validation metrics that is actually a data artifact | POTCAR files are in a shared directory that was updated by a sysadmin; same filename but different content; no checksum was recorded at campaign start | Compute SHA-256 of every POTCAR or UPF file in the dataset directory; compare to the checksum stored in campaign_settings.json; if they differ, the reference energy surface is inconsistent | Record pseudopotential checksums in campaign_settings.json at campaign start (Workflow 1); verify checksums before each DFT batch submission; if a PP file was updated, recompute all structures that used the old version |
| Nondeterministic train/test split | Two colleagues rerunning the same split script on the same dataset produce different train/val/test assignments; test error is unreliable across re-runs | random.shuffle called without random.seed(N); NumPy RNG initialized from system entropy; Pandas DataFrame.sample without random_state | Check the splitting script for any random.seed call; if absent, the split is nondeterministic; compare frame indices across two re-runs of the script | Add random.seed(42) (or equivalent np.random.default_rng(42)) at the top of all splitting scripts; log the seed in the dataset manifest; include the split as a DVC pipeline stage with the seed as a parameter |
| Broken relative paths after cluster migration | DVC pull or MLflow model load fails after moving the project to a new HPC cluster or storage path; model files not found despite being registered | DVC .dvc files or MLflow artifact URIs contain absolute paths hardcoded to the old file system; LAMMPS input scripts have pair_coeff * * /old/path/model.pt | Run grep -r "/old/path" . --include="*.dvc" --include="*.json" --include="*.lmp" to find hardcoded paths; inspect .dvc/config for absolute remote paths | Use relative paths in DVC pipeline stages and LAMMPS scripts; configure DVC remote using environment variables or per-user local config (dvc remote add -d ... --local); test the full pipeline on a fresh clone before campaign handoff |
| Stale model checkpoints — loss of the best model | After re-running training with updated data, the original production model file is gone; only the most recent training run is available | mace_run_train writes to a fixed output filename and overwrites previous runs if --name is the same; no versioned archive was made before re-running | Check MLflow artifacts and model registry; if no artifact was logged, the file is gone unless a backup exists elsewhere | Register every candidate model in MLflow before re-running training; use DVC to track model files alongside datasets (dvc add models/model_v1.model); never run training with the same --name as an existing production model without archiving first |
| Irreproducible figures | A figure in a paper or report cannot be exactly reproduced from the raw data; re-running the plotting script produces different axis ranges, colors, or statistical summaries | Plotting script reads data via glob pattern that changes as new runs are added; matplotlib random state not fixed for stochastic methods; data transformations embedded in the plotting script are not separately versioned | Run the plotting script in a clean environment with only the specified input files; compare output figures pixel-by-pixel using imageio.v3.imread and np.allclose | Pin all input files explicitly (no glob) in figure scripts; seed all random state; version the plotting script alongside the data it visualizes using DVC; log figures as MLflow artifacts immediately after generation |
Best Practices
- Set up Git + DVC before submitting the first DFT calculation. Retroactively adding versioning to an existing dataset is possible but lossy — the history of how the dataset evolved is gone. The setup overhead is less than one DFT calculation's wall time.
- Treat the DFT settings file (
campaign_settings.json) as immutable once committed. Print it, sign it, and enforce the hash check programmatically in every label collection script. A settings mismatch detected later requires recomputing the inconsistent structures, which is expensive. - Use explicit seeds everywhere: Python RNG, NumPy RNG, PyTorch RNG, and LAMMPS velocity initialization. A missing seed is invisible — results appear correct until a collaborator or reviewer cannot reproduce them. Document all seeds in the campaign log.
- Archive every candidate model file before re-running training with the same model name. Storage is cheap; lost model weights require days of GPU time to recover. Use
dvc addor MLflow artifact logging to create versioned archives. - Write provenance sidecars immediately after each computational step, not after the campaign is complete. Retroactive provenance reconstruction from file timestamps and OUTCAR headers is unreliable and cannot recover information that was never recorded.
- Track environment snapshots at campaign start and at every major software upgrade. A
requirements.txtorenvironment.ymlis not sufficient if compiled binaries (LAMMPS, VASP, QE) or CUDA drivers are involved. Record fullpip list,conda env export, and key binary versions. - Use content hashes (SHA-256), not timestamps, to verify file identity. File timestamps are easily corrupted by
touch, rsync, or cluster backup systems. A SHA-256 hash is unforgeable without modifying the file content. - Document the complete training command (not just hyperparameters) as an artifact.
mace_run_trainhas many default values that change between versions; recording only the non-default parameters in a README is insufficient for bit-exact reproduction.
Integration with Other Skills
- dft-dataset-generation: The
DatasetManifestclass in that skill's Workflow 6 is the domain-specific provenance layer for DFT datasets. This skill'sHashRegistryandrecord_campaign_settingsare the campaign-level layer above it. Both must be used together for complete DFT dataset provenance. - dft-convergence: The ENCUT and k-point density established by that skill's convergence protocol must be frozen at campaign start and recorded in
campaign_settings.json. Any convergence re-test that changes the settings hash invalidates all prior data. - vasp-workflow: VASP calculations require versioned POTCAR files (SHA-256), logged INCAR settings, and tracked vasprun.xml outputs. The DFT provenance capture in Workflow 2 above implements the recording layer for VASP. The POTCAR TITEL line and checksum from that skill's best practices are required fields in
campaign_settings.json. - quantum-espresso: QE calculations require pseudopotential filename and checksum provenance (SSSP or PseudoDojo, specific version). The
degauss-in-Ry distinction documented there is a canonical example of a silent settings error that only provenance tracking reveals. - lammps-workflow: LAMMPS production runs must record their input script Git commit hash, model file SHA-256, initial structure file, random seed for velocity initialization, and the LAMMPS version. The environment capture in Workflow 4 covers LAMMPS binary version; the input script belongs in Git.
- md-equilibration: Equilibration runs are often repeated when a new MLP model is trained. Without versioned model files and fixed equilibration seeds, two "identical" equilibration runs with different model checkpoints will produce different structures. Always checkpoint the model SHA-256 alongside the equilibration output.
- mace-training: Workflow 3 above wraps
mace_run_trainwith MLflow tracking. The--seedargument, the training data file checksum, the training command, and the model file checksum are the minimum required to reproduce a MACE training result. The best-practice note in that skill to record the MACE version and model checksum is implemented here viarun_mace_with_tracking. - active-learning-mlp: The
ALCampaignLogin Workflow 6 above is the provenance layer for the active learning loop from that skill. Each AL iteration record connects the committee uncertainty thresholds, the selected DFT structures, the DFT settings hash, and the resulting model version into a single auditable chain. - mlp-validation: Validation results (EOS errors, phonon errors, NVE drift) must be logged as MLflow metrics alongside the model they characterize. A model version can only be moved from
StagingtoProductionin the MLflow model registry after all validation checks from that skill pass. The validation metrics are stored as MLflow run metadata and linked to the model checkpoint. - mlp-to-lammps: Every LAMMPS deployment of an MLP should verify that the deployed model file's SHA-256 matches the registered production model's checksum in the MLflow model registry. The
validate_lammps_forcesfunction in that skill should record the model hash alongside the force comparison result. - materials-featurization: Structural descriptors (SOAP, ACE, MACE latent features) used for diversity filtering and duplicate detection must be versioned alongside the dataset. The SOAP hyperparameters (r_cut, n_max, l_max, species) are part of the preprocessing provenance and must be recorded in the dataset sidecar.
Key References
- Wilkinson, M. D., et al. (2016). The FAIR Guiding Principles for scientific data management and stewardship. Scientific Data, 3, 160018. https://doi.org/10.1038/sdata.2016.18
- Scheffler, M., et al. (2022). FAIR data enabling new horizons for materials research. Nature, 604, 635–642. https://doi.org/10.1038/s41586-022-04501-x
- Huber, S. P., et al. (2021). AiiDA 1.0: A scalable computational infrastructure for automated reproducible workflows and data provenance in computational science. Scientific Data, 7, 300. https://doi.org/10.1038/s41597-020-00638-4
- Pineau, J., et al. (2021). Improving reproducibility in machine learning research (a report from the NeurIPS 2019 reproducibility program). Journal of Machine Learning Research, 22(164), 1–20. https://jmlr.org/papers/v22/20-1364.html
- Draxl, C., & Scheffler, M. (2019). NOMAD: The FAIR concept for big data-driven materials science. MRS Bulletin, 43(9), 676–682. https://doi.org/10.1557/mrs.2018.208
示例提示词
帮我为这批DFT数据集和模型训练记录完整的溯源和版本管理信息。
所属插件
安装该插件会一并获得同领域的 42 个 Skill / MCP。
同插件的其他科研 Skill
- 贝叶斯优化材料发现Covers Bayesian optimization for materials and chemistry design: surrogate models, acquisition functions, batch/constrained BO. Use when planning sample-efficient experiments or simulations.
- CALPHAD热力学计算工作流Covers CALPHAD computational thermodynamics: database selection, equilibrium/metastable calculations, binary/ternary phase diagrams. Use when predicting phase stability or solidification paths.
- 材料第一性原理与分子动力学模拟Routes first-principles and classical materials simulation (DFT, MD, ML potentials) to the right specialized workflow. Use as an entry point for any simulation-based materials task.
- 蠕变力学建模与验证Selects, calibrates, and validates creep deformation and rupture models under sustained thermomechanical load, covering rate prediction. Use when assessing long-term high-temperature components.
- 晶体塑性有限元建模Covers crystal plasticity and CPFE modeling of metals: slip systems for FCC/BCC/HCP, Schmid factors. Use when simulating texture- and orientation-dependent plastic deformation.
- 衍射结构表征Covers diffraction-based structure characterization (XRD, neutron, SAED): space group determination, Rietveld refinement, phase identification. Use for any diffraction-pattern analysis technique.
- EBSD织构分析Analyzes EBSD orientation maps for grain size, misorientation, phase, and texture, with pole figures and twin-boundary identification. Use for electron backscatter diffraction workflows.
- 电化学性能测试分析Covers electrochemical characterization: cyclic voltammetry, battery cycling analysis, EIS. Use for electrochemistry-specific data analysis distinct from general materials characterization.
- 金属疲劳寿命评估Builds, calibrates, and validates fatigue assessments for metals and components: stress-life, strain-life, multiaxial critical-plane. Use when predicting fatigue life or crack growth.
- 多物理场有限元建模Covers finite element modeling for coupled materials, mechanics, and transport problems: weak-form construction, boundary conditions, mesh choice. Use for multiphysics FEM simulations.
- 连续尺度有限元力学建模Covers continuum-scale mechanical, thermal, and multiphysics simulation with Abaqus/COMSOL conventions, mesh generation. Use for macroscale FEM consuming measured or computed material properties.
- 断裂力学评估与建模Selects, constructs, and validates fracture-mechanics assessments: LEFM checks, elastic-plastic fracture mechanics. Use when assessing cracked materials or components.
- 高斯过程材料回归建模Covers Gaussian process regression for materials and chemistry datasets: descriptor selection, kernel design. Use when a probabilistic surrogate model with uncertainty is needed.
- 自驱动实验室闭环材料发现Covers combinatorial synthesis, robotic experimentation. Use for autonomous or robotics-driven experimental campaigns, a less standardized domain.
- 材料数据库可复现访问Covers reproducible API access to materials databases (Materials Project, AFLOW, OQMD, NOMAD, COD, ICSD-style sources): authentication. Use when programmatically querying these databases.
- 材料数据库跨库查询Covers querying public materials databases beyond Materials Project: AFLOW, OQMD, NOMAD, JARVIS. Use when pulling or comparing data from these live, API-integrated resources.
- 材料数据标准与互操作Covers data interoperability standards for materials science: CIF crystal files, the OPTIMADE query API. Use when structuring or exchanging materials data between tools.
- 材料结构特征化Covers converting structures, compositions, and trajectories into ML feature vectors: Magpie-style elemental statistics. Use when building inputs for materials ML models.
- 材料科学图表绘制规范Covers publication-quality figure conventions for materials science plots: phase diagrams, pole figures, stress-strain curves. Use when the deliverable is a figure, not just an analysis result.
- 材料信息学性质预测Covers ML for materials property prediction and generative design: descriptor-based models, crystal GNNs (CGCNN/MEGNet-class). Use for materials informatics distinct from interatomic potentials.
- 材料生命周期评估Covers life-cycle assessment for materials and manufacturing routes: goal/scope definition, functional units, inventory construction. Use for environmental-impact studies of materials.
- 材料科学文献检索Covers formulating literature search queries for materials science questions and synthesizing/citing results responsibly. Requires a web search tool; it cannot substitute for one.
- 工程材料选型设计Selects engineering materials via material indices and Ashby charts, ranking candidates by performance per cost or mass. Use when picking a material class for a design constraint.
- 材料统计方法与实验设计Covers statistics shared across materials characterization and simulation: design of experiments. Use for cross-cutting statistical treatment of materials data.
常见问题
计算材料科学可复现性管理能做什么?
Covers provenance tracking, data versioning, and environment capture for reproducible computational materials campaigns: DFT dataset. Use when documenting a simulation campaign for reuse。
如何安装计算材料科学可复现性管理?
在科搭 Codar 左侧点击「插件」,在「组件」页签搜索「计算材料科学可复现性管理」并安装;在 Claude Code 或 Codex 中可以发送本页的安装提示词,或下载 Skill 包后解压到 ~/.claude/skills 或 ~/.codex/skills。
计算材料科学可复现性管理需要哪些权限?
该科研 Skill没有声明额外权限;执行具体操作时仍由 Agent 按自身规则请求确认。
计算材料科学可复现性管理经过官方审核吗?
是。SpeedSkills 收录的每个版本都会记录内容摘要与权限声明,当前版本已通过安全审核。