科研 Skill
机器学习势产前验证
mlp-validation
Covers systematic validation of ML interatomic potentials before deployment: test-set error and physical property reproduction (EOS, phonons, elastic constants). Use before trusting an MLP.
SKILL.md
MLP Validation
Description
This skill covers the systematic validation of machine-learned interatomic potentials (MLPs) before production deployment: static test-set error analysis, physical property reproduction (equation of state, phonons, elastic constants, surface and defect energies), MD stability testing, extrapolation detection, and deployment consistency verification between training code, ASE, and LAMMPS. Invoke this skill after completing MLP training (see mace-training) and before using the MLP for production MD or property prediction. A low test-set error is necessary but not sufficient for a production-quality MLP — this skill documents the additional checks required to catch the failure modes that pass the training metrics.
Domain Context
An MLP is a function approximator fit to a finite set of DFT-labeled structures. The training loss and test-set error measure how well the model interpolates within the sampled configuration space. They do not measure:
- Whether the sampled space is representative of the target application (MD at a given temperature and pressure, surface chemistry, defect migration)
- Whether the model produces physically correct macroscopic properties (thermal expansion, phonon frequencies, elastic moduli) that emerge from the collective behavior of the potential energy surface
- Whether the model is numerically stable over nanosecond timescales, where rare high-energy configurations may be sampled with non-negligible probability
- Whether the LAMMPS and ASE implementations of the same model file produce identical forces (they should, but unit conversions and element-order conventions have introduced silent bugs in practice)
The validation trinity structures this skill:
- Static errors: Energy, force, and stress MAE on held-out DFT structures, decomposed by chemical environment and configuration family. This is the fastest check and narrows down systematic biases.
- Physical property reproduction: Comparison of MLP-predicted properties against DFT references computed at the same level of theory. EOS, phonon frequencies, elastic constants, and defect energies are the most diagnostic. These catch errors in the many-body and stress response that per-atom force errors miss.
- MD stability: Short NVE energy conservation runs, NVT temperature stability, and NPT density and structural reproduction. MD integrates the potential repeatedly and amplifies any nonphysical region of the PES that was not covered by training.
Reference standard: MLP predictions must be compared against DFT, not against another MLP. Benchmarking one MLP against another (e.g., comparing MACE to a Behler–Parrinello network on the same test set) measures model agreement, not physical accuracy. The DFT level used for training data is the correct reference for all validation checks.
Held-out test set integrity: The test set used for final validation must be completely isolated from the training process, the active learning query strategy, and the hyperparameter search. Structures that informed any training decision — including model architecture selection — are contaminated for accuracy estimation purposes.
When to Use This Skill
- After training any MLP (MACE, NequIP, Allegro, DeepMD, GAP) and before production deployment.
- When an active learning campaign signals convergence and a final model is being commissioned.
- When an existing MLP is being extended to new conditions (new temperature range, new phase, surface chemistry) and the domain extension must be validated.
- When an MLP from the literature is being reused on a new system and its transferability must be verified.
- When deploying to a new code (switching from ASE to LAMMPS, or upgrading MACE version) and deployment consistency must be confirmed.
- When quantifying the reliability of MLP-predicted properties for a paper or report.
When Not to Use This Skill
- As a substitute for active learning. If the model fails validation, the correct response is to expand the training dataset (see
active-learning-mlp), not to adjust validation thresholds. - When the model has not yet converged. Training-run validation metrics should be monitored during training via
mace-trainingWorkflow 4. This skill is for post-training, post-convergence validation. - For properties outside the Born-Oppenheimer surface — electronic excitations, photochemistry, non-adiabatic dynamics. The MLP models the ground-state PES; no validation on this surface certifies accuracy on excited states.
- As the sole assessment for high-stakes applications. For safety-critical or regulatory applications (e.g., nuclear materials, pharmaceutical molecular dynamics) MLP validation is a necessary but not sufficient quality assurance step; additional independent expert review and experimental benchmarks are required.
Core Concepts
- Validation trinity: Static errors + physical property reproduction + MD stability. All three are required; passing one does not compensate for failing another.
- Per-config-type decomposition: Global test-set MAE conceals systematic failures in minority or critical configuration families. A model with 30 meV/Å global force MAE may have 200 meV/Å for surface configurations if surfaces are 5% of the test set. Always report errors broken down by
config_type. - Per-species error decomposition: For multi-element systems, force errors per chemical species reveal gaps in chemical coverage. A high force error for a low-abundance species (e.g., a dopant) indicates insufficient training coverage of that species' bonding environments.
- Learning curve: A plot of test-set error (force MAE) as a function of training set size. A flat learning curve at the current training size indicates that data quantity is no longer the bottleneck; a still-decreasing curve indicates more data would improve accuracy. Computed by training sub-models on nested subsets of the training data.
- Equation of state (EOS): Energy vs. volume curve for a bulk structure, fit to the Birch-Murnaghan or Vinet equation. MLP should reproduce DFT equilibrium volume V₀, bulk modulus B₀, and pressure derivative B₀' within specified tolerances. EOS errors are the most sensitive indicator of stress tensor quality.
- Phonon spectrum: Normal mode frequencies from the harmonic approximation, computed via finite displacements (phonopy) or DFPT. MLP phonon errors are typically dominated by the accuracy of forces on symmetry-displaced supercell structures. Imaginary modes in stable crystals indicate a locally unstable PES.
- Elastic constants (Cᵢⱼ): Second derivatives of the strain energy with respect to applied strain. Computed by applying uniform strains and fitting the energy-strain curve. Sensitive to the stress response of the MLP — elastic constants are the most stringent test of stress label accuracy.
- Radial distribution function (RDF): Pair correlation function g(r) from MD trajectory. Comparison of MLP-MD RDF to DFT-MD or experimental data validates structural properties at finite temperature. Peak positions should agree within 0.05 Å; peak heights within ~10%. [EXPERT REVIEW NEEDED]
- NVE energy drift: Rate of total energy drift in microcanonical MD. Drift arises from numerical integration error (timestep too large), unphysical configurations in the trajectory (model extrapolation), or implementation bugs. Acceptable drift is system-dependent but typically < 1 meV/atom/ps. [EXPERT REVIEW NEEDED]
- Deployment consistency: The same MACE model file should produce bitwise-identical forces when called from MACECalculator (ASE), from LAMMPS pair_style mace, and from MACE's own evaluation script. Discrepancies above 1 meV/Å indicate element-order mismatches, unit errors, or precision differences.
- Extrapolation detection: Uncertainty estimation using committee models (force std across N trained models) or structural dissimilarity metrics (SOAP distances to the training set). A structure is flagged for potential extrapolation if its committee σ_F exceeds a calibrated threshold or if its minimum SOAP distance to the training set exceeds a maximum-coverage radius.
Key Workflows
Full steps for each part are in references/key-workflows/; open the one you need:
- Workflow 1: Static error analysis — energy, force, and stress decomposition —
references/key-workflows/01-workflow-1-static-error-analysis-energy-force-an.md: The baseline check. Run before any physical property validation. Identify whether errors are globally acceptable and which configuration families or species are problematic. - Workflow 2: Learning curves and data sufficiency —
references/key-workflows/02-workflow-2-learning-curves-and-data-sufficiency.md: Determines whether the training set is large enough or whether more DFT data would meaningfully reduce errors. Trains a sequence of sub-models on nested subsets of increasing size. - Workflow 3: Equation of state and elastic constant validation —
references/key-workflows/03-workflow-3-equation-of-state-and-elastic-constan.md: The EOS validates the bulk modulus, equilibrium volume, and pressure response — the most sensitive tests of stress tensor accuracy. Elastic constants are the most stringent check: they require accurate second derivatives of the energy with… - Workflow 4: Phonon validation using finite displacements —
references/key-workflows/04-workflow-4-phonon-validation-using-finite-displa.md: Phonon frequencies are among the most sensitive tests of the local curvature of the MLP PES. Imaginary modes for a known stable crystal are a definitive sign of a locally incorrect potential. - Workflow 5: MD stability — NVE conservation, NVT, and RDF comparison —
references/key-workflows/05-workflow-5-md-stability-nve-conservation-nvt-and.md: MD stability tests reveal whether the MLP is usable for the target application conditions. Failures here are not fixed by retraining hyperparameters — they require expanding the training set via active learning. - Workflow 6: Deployment consistency — ASE vs. LAMMPS force comparison —
references/key-workflows/06-workflow-6-deployment-consistency-ase-vs-lammps.md: The same model file must produce identical forces in ASE and LAMMPS. This check must pass before trusting LAMMPS production runs. Element-type ordering, units, and floating-point precision are all sources of silent discrepancies.
Common Failure Modes
| Failure | Symptom | Likely Cause | Diagnostic | Fix |
|---|---|---|---|---|
| Low test error but unstable MD | Test force MAE < 50 meV/Å but MD diverges within 10 ps; energy explosion; max force > 100 eV/Å during MD | Training structures do not cover the dynamically accessible configurations at the target temperature or density; high-energy region of PES is unphysical | Monitor max force during 5 ps NVE test run at target temperature; if any frame exceeds 30 eV/Å the model is extrapolating | Run active learning at the target temperature and conditions; add rattled structures with larger displacements; expand the DFT temperature range |
| Wrong stress sign — NPT box explosion or collapse | NPT MD box volume diverges monotonically; density wrong by a factor of 2 or more; bulk modulus from EOS has wrong sign | Stress labels in training set have wrong sign convention (VASP compression-positive not converted to ASE tensile-positive) | Check equilibrium structure: ASE stress diagonal should be ≈ 0 and positive for a compressed cell; if all components have wrong sign, convention is inverted | Apply stress = -kBar * KBAR_TO_EV_A3 in the DFT conversion (see mace-training Workflow 1); recompute stress for all training structures |
| Missing chemistry — garbage forces for one species | Forces on one atom type are physically impossible (> 10 eV/Å in equilibrium geometry); energy wrong by > 1 eV/atom for any structure containing that species | Species is absent or nearly absent from the training set; or species appears only in one bonding environment | Report per-species force MAE (Workflow 1); count training frames per species; check species in every config_type | Add DFT calculations covering the missing species in all relevant bonding environments (bulk, surface, defect, alloy neighbor) |
| Poor high-energy behavior — energy catastrophe | Energy correct near equilibrium but diverges sharply at bond compression (< 0.8 × equilibrium bond length); reactive MD or NPT at high P fails | Training data does not cover short interatomic distances; MLP predicts unphysical attractive forces at short range | Sample g(r) during MD; compute energy vs bond length for a dimer at sub-equilibrium separations and compare to DFT | Include structures with compressed bonds in training (volumetric strain series ± 15%); use ZBL-style short-range correction if available for the architecture |
| Data leakage — optimistic test error | Test force MAE appears very low (< 10 meV/Å) but production MD fails quickly; active learning loops overestimate convergence | Adjacent frames from the same MD trajectory split randomly into train and test; near-identical structures in both splits | Compute SOAP distance between every test frame and its nearest training neighbor; if > 50% of test frames have SOAP distance < 0.001, test set is contaminated | Re-split by trajectory or by rattled-origin (see mace-training Workflow 2); ensure test set contains only structures drawn from distinct initial conditions |
| Bad unit conversion — systematic energy offset | Energy MAE is > 0.5 eV/atom despite correct forces; formation energies wrong by integer multiples of a known factor (×13.6 = Ry→eV, ×27.2 = Ha→eV, ×29.3 = Ry/bohr³ stress) | Energies or forces written in wrong units (Hartree or Rydberg instead of eV); distances in bohr instead of Å; stress in Ry/bohr³ | Check raw extxyz energy values: for metals, E/atom should be −4 to −12 eV/atom; for oxides −5 to −10 eV/atom; flag outliers ×13.6 or ×27.2 off | Convert all labels to eV and Å before writing extxyz; use ASE units.Ry and units.Bohr conversion constants |
| False confidence outside training domain | Committee uncertainty is uniformly low; model reports low σ_F for structures it has never seen; production fails with no uncertainty warning | Committee members have insufficient diversity — same architecture, same data, nearly same random seed; model has memorized the training distribution | Deliberately query a structure well outside the training domain (e.g., 2× compressed unit cell) and check σ_F; if σ_F < σ_low, the committee is overconfident | Increase committee diversity: larger seed separation, jackknife data subsets, or different architectures; calibrate thresholds using held-out out-of-distribution structures |
| EOS curvature wrong — bulk modulus error > 10% | Phonon frequencies systematically too high or too low; thermal expansion coefficient wrong sign or magnitude; NPT density drifts slowly | Stress tensor errors in training data; stress labels computed at different ENCUT or without converged basis; or stress labels excluded from training | Compute EOS at 11 volumes spanning ±10% and compare to DFT; check if stress labels are present in training extxyz (stress key in atoms.info) | Verify stress labels with verify_stress_sign on equilibrium structure; ensure --stress_weight > 0 in training; recompute stress if ENCUT differs from bulk convergence run |
Best Practices
- Validate against DFT, never against another MLP. Comparing MACE to a classical force field or another neural network measures model agreement, not physical accuracy. Every validation check in this skill requires a DFT reference.
- Maintain a clean held-out test set from Iteration 0. Structures targeted by the active learning query strategy are contaminated for unbiased accuracy estimation. A clean test set — assembled before active learning begins, never shown to the model — is the only reliable accuracy gauge across all iterations.
- Always use float64 precision for validation. Float32 can mask energy errors below 1 meV/atom and force errors below 5 meV/Å. Enable
default_dtype="float64"in MACECalculator for all validation evaluations even if training used float32. - Run the ASE vs. LAMMPS consistency check before any LAMMPS production run. Element type ordering, the
pair_coeffelement list, and theno_domain_decompositionflag must all be correct. A silent error here produces wrong forces with no error message. - Check per-species and per-config-type errors, not just the global MAE. A 30 meV/Å global MAE may hide 300 meV/Å errors for a dopant species or a surface configuration that appears in < 5% of the test set. Report and inspect every config_type separately.
- Run the NVE energy conservation test at the target temperature before committing to NPT. A drift > 1 meV/atom/ps at target temperature is a contraindication for production MD. Reduce the timestep first (try 1 fs); if drift persists, the model needs active learning at that temperature.
- Validate EOS before running NPT. The NPT barostat uses forces and stresses to adjust the cell. A wrong stress sign or a B₀ error > 20% will cause unphysical density behavior. Run Workflow 3 and confirm V₀ and B₀ before enabling pressure coupling.
- Check for imaginary phonon modes at Γ. Even one imaginary mode in a nominally stable crystal is a definitive indicator of a locally incorrect MLP PES. Compute Γ-point phonons at negligible cost and flag any imaginary frequency < −0.1 THz.
- Document the validation suite results in the project README with model checksum, MACE version, test set size and composition, and achieved metrics. This is the minimum reproducibility record for MLP deployment.
- For surfaces, defects, and reactions, validate each separately. Bulk validation passing does not certify surface or defect accuracy. Include per-environment validation whenever the target application involves environments that are structurally distinct from bulk. [EXPERT REVIEW NEEDED]
Integration with Other Skills
- mace-training: This skill follows mace-training in the MLP development workflow. The test set from that skill's stratified split is the primary input to Workflow 1. The stress sign convention and float64 validation prescription from mace-training carry directly into this skill.
- active-learning-mlp: Workflow 6 here (NVE stability test and extrapolation coverage check) is the pre-deployment gate before declaring active learning convergence. When validation fails, the active learning loop must be resumed with the failing structures as targets for new DFT labels.
- mlp-to-lammps: Workflow 6 (ASE vs. LAMMPS consistency check) verifies the LAMMPS deployment described in detail in that skill. The element-type ordering requirements and
no_domain_decompositionflag covered there directly affect the consistency check. - uncertainty-mlp: The committee-based extrapolation coverage check in Workflow 6 is the simplest uncertainty quantification approach. That skill covers principled alternatives (conformal prediction, last-layer Bayesian estimation) and when calibration of uncertainty estimates is required beyond the σ_F threshold approach.
- dft-dataset-generation: When validation reveals systematic failures (missing config_type, wrong chemistry for a species, poor surface accuracy), the remedy is always to generate more targeted DFT data. That skill covers how to design the supplementary DFT calculations.
- dft-convergence: DFT reference calculations for phonon, EOS, and elastic constant validation must be computed at the same level of theory as the training data. If a discrepancy appears between MLP and DFT property references, verify that the reference DFT settings are identical to those used for training before concluding the MLP is wrong.
- vasp-workflow: DFT reference calculations for EOS, phonon, and elastic property validation are typically run with VASP. The ISIF settings (ISIF=3 for cell relaxation, ISIF=2 for stress at fixed cell) from that skill are required for computing accurate reference stress tensors and elastic constants.
- quantum-espresso: When QE is the reference code for validation, phonopy can use QE force outputs directly. Workflow 4's finite displacement approach applies identically; only the DFT interface changes.
- lammps-workflow: The NVE and NVT MD stability tests in Workflow 5 can be run in LAMMPS as well as ASE. The equilibration protocol, timestep conventions, and thermostat settings from that skill apply to MACE-driven LAMMPS validation runs.
- md-equilibration: The pre-validation equilibration of the starting structure for NVE tests must follow the protocol from that skill. Starting from a poorly equilibrated structure inflates the apparent energy drift and produces a pessimistic NVE assessment.
- dft-phonons: The DFT reference phonon calculation that MLP phonons are validated against is described in that skill. The supercell construction, displacement protocol, and BORN charge handling for polar materials described there must match the phonopy setup in Workflow 4.
- eos-fitting: The Birch-Murnaghan EOS fitting and the DFT reference EOS required for Workflow 3 follow the protocol in that skill. When DFT and MLP EOS are computed consistently, discrepancies in V₀ and B₀ are unambiguously attributable to the MLP.
Key References
- Zuo, Y., et al. (2020). Performance and cost assessment of machine learning interatomic potentials. The Journal of Physical Chemistry A, 124(4), 731–745. https://doi.org/10.1021/acs.jpca.9b08723
- Behler, J. (2016). Perspective: Machine learning potentials for atomistic simulations. The Journal of Chemical Physics, 145(17), 170901. https://doi.org/10.1063/1.4966192
- Musil, F., et al. (2021). Physics-inspired structural representations for molecules and materials. Chemical Reviews, 121(16), 9759–9815. https://doi.org/10.1021/acs.chemrev.1c00021
- Miksch, A. M., et al. (2021). Strategies for the construction of machine-learning potentials for accurate and efficient atomic-scale simulations. Machine Learning: Science and Technology, 2(3), 031001. https://doi.org/10.1088/2632-2153/abfd96
- Fu, X., et al. (2023). Forces are not enough: Benchmark and critical evaluation of machine learning force fields for molecular simulations. Transactions on Machine Learning Research. https://openreview.net/forum?id=A8pqQipwkt
示例提示词
我想在部署前系统验证这个机器学习势的弹性常数和MD稳定性。
所属插件
安装该插件会一并获得同领域的 35 个 Skill / MCP。
同插件的其他科研 Skill
- 主动学习构建机器学习势训练集Runs active-learning workflows for ML interatomic potentials: uncertainty-guided MD exploration, DFT oracle labeling, query strategy. Use when expanding an MLP training set.
- AiiDA计算工作流管理Covers AiiDA, a provenance-first Python workflow engine for computational materials science: calculation input generation, remote submission. Use when orchestrating reproducible DFT/MD campaigns.
- ASE原子模拟环境Covers the Atomic Simulation Environment (ASE) as an atomistic workflow hub: Atoms objects, calculators, optimizers, trajectories. Use when scripting or converting structures across DFT/MD codes.
- 粗粒化分子动力学模拟Covers coarse-grained molecular dynamics: mapping atomistic structures to CG beads, parameterizing Martini/IBI/force-matching force fields. Use when atomistic timescales are too short.
- CP2K第一性原理分子动力学Covers end-to-end DFT, AIMD, and atomistic workflows with CP2K using the Gaussian-and-plane-waves method: input construction, convergence testing. Use for CP2K-based simulations.
- 扩散模型生成晶体结构Covers de novo crystal structure generation with score-based diffusion and flow-matching models (MatterGen, DiffCSP, CDVAE). Use when generating candidate structures, not analyzing existing ones.
- 晶体图神经网络性质预测Covers graph neural networks for crystal and periodic-material property prediction: crystal graph construction, periodic neighbor finding. Use when building GNN property predictors for crystals.
- 平面波DFT收敛性测试Covers systematic convergence testing for plane-wave DFT: cutoff energy, k-point mesh density, smearing. Use before trusting DFT energies, forces, or derived properties as converged.
- 机器学习势DFT数据集生成Covers generating DFT-labeled datasets for training ML interatomic potentials: configuration-space coverage. Use when building an MLP training dataset from scratch.
- 点缺陷第一性原理计算Covers first-principles point-defect modeling: vacancy/interstitial/antisite/substitutional construction, charge-state calculations. Use for defect thermodynamics studies.
- 第一性原理声子谱计算Covers lattice dynamics from first principles: phonon dispersion, phonon DOS. Use when computing vibrational or thermodynamic properties from DFT.
- 表面模型DFT计算Covers surface slab model construction and DFT calculation: Miller-index generation, termination selection, slab/vacuum convergence. Use for surface science and catalysis DFT studies.
- 弹性常数计算工作流Computes single-crystal and polycrystalline elastic constants from DFT or MD stress-strain data and checks mechanical stability. Use when deriving stiffness tensors or bulk/shear modulus.
- 增强采样自由能计算Covers enhanced sampling methods for atomistic MD (metadynamics, umbrella sampling. Use for rare-event or slow-transition sampling.
- 状态方程拟合Covers equation-of-state fitting from DFT/ML-potential energy-volume data: Birch-Murnaghan, Vinet, and Murnaghan forms. Use when fitting an EOS to compression data.
- 机器学习势微调迁移训练Covers fine-tuning pretrained universal ML interatomic potentials on system-specific DFT data. Use when adapting a universal MLP to a new system.
- 分子动力学自由能计算Covers quantitative free-energy calculations from MD: alchemical FEP/TI/BAR/MBAR, umbrella-sampling PMF. Use when computing thermodynamic free energies from simulation.
- GROMACS分子动力学工作流Covers end-to-end MD simulations with GROMACS: topology/.mdp construction, energy minimization, NVT/NPT equilibration. Use for GROMACS-based biomolecular or materials MD.
- 高通量DFT自动化筛选Covers automated large-scale DFT screening: workflow graphs with atomate2/jobflow/AiiDA, input-set standardization, custodian error handling. Use for high-throughput DFT screening campaigns.
- LAMMPS分子动力学工作流Covers end-to-end MD simulations with LAMMPS: input script anatomy, force-field deployment, ensemble control, equilibration. Use for LAMMPS-based classical molecular dynamics.
- MACE机器学习势训练Covers training MACE machine-learned interatomic potentials: extxyz dataset preparation, isolated-atom energies, train/val/test splitting. Use for MACE-specific MLP training workflows.
- MatGL图神经网络框架Covers building, applying, validating, and fine-tuning MatGL graph-neural-network models for property prediction and interatomic potentials. Use for pretrained GNN workflows via MatGL.
- 分子动力学轨迹后处理分析Covers quantitative post-processing of atomistic MD trajectories: loading LAMMPS/GROMACS/ASE trajectories and computing RDF, coordination numbers. Use for structural MD trajectory analysis.
- 分子动力学平衡过程设计Covers bringing an atomistic simulation from its initial configuration to a representative equilibrium state before production statistics: ensemble. Use before collecting production MD data.
常见问题
机器学习势产前验证能做什么?
Covers systematic validation of ML interatomic potentials before deployment: test-set error and physical property reproduction (EOS, phonons, elastic constants). Use before trusting an MLP。
如何安装机器学习势产前验证?
在科搭 Codar 左侧点击「插件」,在「组件」页签搜索「机器学习势产前验证」并安装;在 Claude Code 或 Codex 中可以发送本页的安装提示词,或下载 Skill 包后解压到 ~/.claude/skills 或 ~/.codex/skills。
机器学习势产前验证需要哪些权限?
该科研 Skill没有声明额外权限;执行具体操作时仍由 Agent 按自身规则请求确认。
机器学习势产前验证经过官方审核吗?
是。SpeedSkills 收录的每个版本都会记录内容摘要与权限声明,当前版本已通过安全审核。