科研 Skill

机器学习势产前验证

mlp-validation

Covers systematic validation of ML interatomic potentials before deployment: test-set error and physical property reproduction (EOS, phonons, elastic constants). Use before trusting an MLP.

发布者 Codar安全审核通过v0.1.0更新于 2026-11-18

SKILL.md

MLP Validation

Description

This skill covers the systematic validation of machine-learned interatomic potentials (MLPs) before production deployment: static test-set error analysis, physical property reproduction (equation of state, phonons, elastic constants, surface and defect energies), MD stability testing, extrapolation detection, and deployment consistency verification between training code, ASE, and LAMMPS. Invoke this skill after completing MLP training (see mace-training) and before using the MLP for production MD or property prediction. A low test-set error is necessary but not sufficient for a production-quality MLP — this skill documents the additional checks required to catch the failure modes that pass the training metrics.

Domain Context

An MLP is a function approximator fit to a finite set of DFT-labeled structures. The training loss and test-set error measure how well the model interpolates within the sampled configuration space. They do not measure:

  • Whether the sampled space is representative of the target application (MD at a given temperature and pressure, surface chemistry, defect migration)
  • Whether the model produces physically correct macroscopic properties (thermal expansion, phonon frequencies, elastic moduli) that emerge from the collective behavior of the potential energy surface
  • Whether the model is numerically stable over nanosecond timescales, where rare high-energy configurations may be sampled with non-negligible probability
  • Whether the LAMMPS and ASE implementations of the same model file produce identical forces (they should, but unit conversions and element-order conventions have introduced silent bugs in practice)

The validation trinity structures this skill:

  1. Static errors: Energy, force, and stress MAE on held-out DFT structures, decomposed by chemical environment and configuration family. This is the fastest check and narrows down systematic biases.
  2. Physical property reproduction: Comparison of MLP-predicted properties against DFT references computed at the same level of theory. EOS, phonon frequencies, elastic constants, and defect energies are the most diagnostic. These catch errors in the many-body and stress response that per-atom force errors miss.
  3. MD stability: Short NVE energy conservation runs, NVT temperature stability, and NPT density and structural reproduction. MD integrates the potential repeatedly and amplifies any nonphysical region of the PES that was not covered by training.

Reference standard: MLP predictions must be compared against DFT, not against another MLP. Benchmarking one MLP against another (e.g., comparing MACE to a Behler–Parrinello network on the same test set) measures model agreement, not physical accuracy. The DFT level used for training data is the correct reference for all validation checks.

Held-out test set integrity: The test set used for final validation must be completely isolated from the training process, the active learning query strategy, and the hyperparameter search. Structures that informed any training decision — including model architecture selection — are contaminated for accuracy estimation purposes.

When to Use This Skill

  • After training any MLP (MACE, NequIP, Allegro, DeepMD, GAP) and before production deployment.
  • When an active learning campaign signals convergence and a final model is being commissioned.
  • When an existing MLP is being extended to new conditions (new temperature range, new phase, surface chemistry) and the domain extension must be validated.
  • When an MLP from the literature is being reused on a new system and its transferability must be verified.
  • When deploying to a new code (switching from ASE to LAMMPS, or upgrading MACE version) and deployment consistency must be confirmed.
  • When quantifying the reliability of MLP-predicted properties for a paper or report.

When Not to Use This Skill

  • As a substitute for active learning. If the model fails validation, the correct response is to expand the training dataset (see active-learning-mlp), not to adjust validation thresholds.
  • When the model has not yet converged. Training-run validation metrics should be monitored during training via mace-training Workflow 4. This skill is for post-training, post-convergence validation.
  • For properties outside the Born-Oppenheimer surface — electronic excitations, photochemistry, non-adiabatic dynamics. The MLP models the ground-state PES; no validation on this surface certifies accuracy on excited states.
  • As the sole assessment for high-stakes applications. For safety-critical or regulatory applications (e.g., nuclear materials, pharmaceutical molecular dynamics) MLP validation is a necessary but not sufficient quality assurance step; additional independent expert review and experimental benchmarks are required.

Core Concepts

  • Validation trinity: Static errors + physical property reproduction + MD stability. All three are required; passing one does not compensate for failing another.
  • Per-config-type decomposition: Global test-set MAE conceals systematic failures in minority or critical configuration families. A model with 30 meV/Å global force MAE may have 200 meV/Å for surface configurations if surfaces are 5% of the test set. Always report errors broken down by config_type.
  • Per-species error decomposition: For multi-element systems, force errors per chemical species reveal gaps in chemical coverage. A high force error for a low-abundance species (e.g., a dopant) indicates insufficient training coverage of that species' bonding environments.
  • Learning curve: A plot of test-set error (force MAE) as a function of training set size. A flat learning curve at the current training size indicates that data quantity is no longer the bottleneck; a still-decreasing curve indicates more data would improve accuracy. Computed by training sub-models on nested subsets of the training data.
  • Equation of state (EOS): Energy vs. volume curve for a bulk structure, fit to the Birch-Murnaghan or Vinet equation. MLP should reproduce DFT equilibrium volume V₀, bulk modulus B₀, and pressure derivative B₀' within specified tolerances. EOS errors are the most sensitive indicator of stress tensor quality.
  • Phonon spectrum: Normal mode frequencies from the harmonic approximation, computed via finite displacements (phonopy) or DFPT. MLP phonon errors are typically dominated by the accuracy of forces on symmetry-displaced supercell structures. Imaginary modes in stable crystals indicate a locally unstable PES.
  • Elastic constants (Cᵢⱼ): Second derivatives of the strain energy with respect to applied strain. Computed by applying uniform strains and fitting the energy-strain curve. Sensitive to the stress response of the MLP — elastic constants are the most stringent test of stress label accuracy.
  • Radial distribution function (RDF): Pair correlation function g(r) from MD trajectory. Comparison of MLP-MD RDF to DFT-MD or experimental data validates structural properties at finite temperature. Peak positions should agree within 0.05 Å; peak heights within ~10%. [EXPERT REVIEW NEEDED]
  • NVE energy drift: Rate of total energy drift in microcanonical MD. Drift arises from numerical integration error (timestep too large), unphysical configurations in the trajectory (model extrapolation), or implementation bugs. Acceptable drift is system-dependent but typically < 1 meV/atom/ps. [EXPERT REVIEW NEEDED]
  • Deployment consistency: The same MACE model file should produce bitwise-identical forces when called from MACECalculator (ASE), from LAMMPS pair_style mace, and from MACE's own evaluation script. Discrepancies above 1 meV/Å indicate element-order mismatches, unit errors, or precision differences.
  • Extrapolation detection: Uncertainty estimation using committee models (force std across N trained models) or structural dissimilarity metrics (SOAP distances to the training set). A structure is flagged for potential extrapolation if its committee σ_F exceeds a calibrated threshold or if its minimum SOAP distance to the training set exceeds a maximum-coverage radius.

Key Workflows

Full steps for each part are in references/key-workflows/; open the one you need:

Common Failure Modes

FailureSymptomLikely CauseDiagnosticFix
Low test error but unstable MDTest force MAE < 50 meV/Å but MD diverges within 10 ps; energy explosion; max force > 100 eV/Å during MDTraining structures do not cover the dynamically accessible configurations at the target temperature or density; high-energy region of PES is unphysicalMonitor max force during 5 ps NVE test run at target temperature; if any frame exceeds 30 eV/Å the model is extrapolatingRun active learning at the target temperature and conditions; add rattled structures with larger displacements; expand the DFT temperature range
Wrong stress sign — NPT box explosion or collapseNPT MD box volume diverges monotonically; density wrong by a factor of 2 or more; bulk modulus from EOS has wrong signStress labels in training set have wrong sign convention (VASP compression-positive not converted to ASE tensile-positive)Check equilibrium structure: ASE stress diagonal should be ≈ 0 and positive for a compressed cell; if all components have wrong sign, convention is invertedApply stress = -kBar * KBAR_TO_EV_A3 in the DFT conversion (see mace-training Workflow 1); recompute stress for all training structures
Missing chemistry — garbage forces for one speciesForces on one atom type are physically impossible (> 10 eV/Å in equilibrium geometry); energy wrong by > 1 eV/atom for any structure containing that speciesSpecies is absent or nearly absent from the training set; or species appears only in one bonding environmentReport per-species force MAE (Workflow 1); count training frames per species; check species in every config_typeAdd DFT calculations covering the missing species in all relevant bonding environments (bulk, surface, defect, alloy neighbor)
Poor high-energy behavior — energy catastropheEnergy correct near equilibrium but diverges sharply at bond compression (< 0.8 × equilibrium bond length); reactive MD or NPT at high P failsTraining data does not cover short interatomic distances; MLP predicts unphysical attractive forces at short rangeSample g(r) during MD; compute energy vs bond length for a dimer at sub-equilibrium separations and compare to DFTInclude structures with compressed bonds in training (volumetric strain series ± 15%); use ZBL-style short-range correction if available for the architecture
Data leakage — optimistic test errorTest force MAE appears very low (< 10 meV/Å) but production MD fails quickly; active learning loops overestimate convergenceAdjacent frames from the same MD trajectory split randomly into train and test; near-identical structures in both splitsCompute SOAP distance between every test frame and its nearest training neighbor; if > 50% of test frames have SOAP distance < 0.001, test set is contaminatedRe-split by trajectory or by rattled-origin (see mace-training Workflow 2); ensure test set contains only structures drawn from distinct initial conditions
Bad unit conversion — systematic energy offsetEnergy MAE is > 0.5 eV/atom despite correct forces; formation energies wrong by integer multiples of a known factor (×13.6 = Ry→eV, ×27.2 = Ha→eV, ×29.3 = Ry/bohr³ stress)Energies or forces written in wrong units (Hartree or Rydberg instead of eV); distances in bohr instead of Å; stress in Ry/bohr³Check raw extxyz energy values: for metals, E/atom should be −4 to −12 eV/atom; for oxides −5 to −10 eV/atom; flag outliers ×13.6 or ×27.2 offConvert all labels to eV and Å before writing extxyz; use ASE units.Ry and units.Bohr conversion constants
False confidence outside training domainCommittee uncertainty is uniformly low; model reports low σ_F for structures it has never seen; production fails with no uncertainty warningCommittee members have insufficient diversity — same architecture, same data, nearly same random seed; model has memorized the training distributionDeliberately query a structure well outside the training domain (e.g., 2× compressed unit cell) and check σ_F; if σ_F < σ_low, the committee is overconfidentIncrease committee diversity: larger seed separation, jackknife data subsets, or different architectures; calibrate thresholds using held-out out-of-distribution structures
EOS curvature wrong — bulk modulus error > 10%Phonon frequencies systematically too high or too low; thermal expansion coefficient wrong sign or magnitude; NPT density drifts slowlyStress tensor errors in training data; stress labels computed at different ENCUT or without converged basis; or stress labels excluded from trainingCompute EOS at 11 volumes spanning ±10% and compare to DFT; check if stress labels are present in training extxyz (stress key in atoms.info)Verify stress labels with verify_stress_sign on equilibrium structure; ensure --stress_weight > 0 in training; recompute stress if ENCUT differs from bulk convergence run

Best Practices

  • Validate against DFT, never against another MLP. Comparing MACE to a classical force field or another neural network measures model agreement, not physical accuracy. Every validation check in this skill requires a DFT reference.
  • Maintain a clean held-out test set from Iteration 0. Structures targeted by the active learning query strategy are contaminated for unbiased accuracy estimation. A clean test set — assembled before active learning begins, never shown to the model — is the only reliable accuracy gauge across all iterations.
  • Always use float64 precision for validation. Float32 can mask energy errors below 1 meV/atom and force errors below 5 meV/Å. Enable default_dtype="float64" in MACECalculator for all validation evaluations even if training used float32.
  • Run the ASE vs. LAMMPS consistency check before any LAMMPS production run. Element type ordering, the pair_coeff element list, and the no_domain_decomposition flag must all be correct. A silent error here produces wrong forces with no error message.
  • Check per-species and per-config-type errors, not just the global MAE. A 30 meV/Å global MAE may hide 300 meV/Å errors for a dopant species or a surface configuration that appears in < 5% of the test set. Report and inspect every config_type separately.
  • Run the NVE energy conservation test at the target temperature before committing to NPT. A drift > 1 meV/atom/ps at target temperature is a contraindication for production MD. Reduce the timestep first (try 1 fs); if drift persists, the model needs active learning at that temperature.
  • Validate EOS before running NPT. The NPT barostat uses forces and stresses to adjust the cell. A wrong stress sign or a B₀ error > 20% will cause unphysical density behavior. Run Workflow 3 and confirm V₀ and B₀ before enabling pressure coupling.
  • Check for imaginary phonon modes at Γ. Even one imaginary mode in a nominally stable crystal is a definitive indicator of a locally incorrect MLP PES. Compute Γ-point phonons at negligible cost and flag any imaginary frequency < −0.1 THz.
  • Document the validation suite results in the project README with model checksum, MACE version, test set size and composition, and achieved metrics. This is the minimum reproducibility record for MLP deployment.
  • For surfaces, defects, and reactions, validate each separately. Bulk validation passing does not certify surface or defect accuracy. Include per-environment validation whenever the target application involves environments that are structurally distinct from bulk. [EXPERT REVIEW NEEDED]

Integration with Other Skills

  • mace-training: This skill follows mace-training in the MLP development workflow. The test set from that skill's stratified split is the primary input to Workflow 1. The stress sign convention and float64 validation prescription from mace-training carry directly into this skill.
  • active-learning-mlp: Workflow 6 here (NVE stability test and extrapolation coverage check) is the pre-deployment gate before declaring active learning convergence. When validation fails, the active learning loop must be resumed with the failing structures as targets for new DFT labels.
  • mlp-to-lammps: Workflow 6 (ASE vs. LAMMPS consistency check) verifies the LAMMPS deployment described in detail in that skill. The element-type ordering requirements and no_domain_decomposition flag covered there directly affect the consistency check.
  • uncertainty-mlp: The committee-based extrapolation coverage check in Workflow 6 is the simplest uncertainty quantification approach. That skill covers principled alternatives (conformal prediction, last-layer Bayesian estimation) and when calibration of uncertainty estimates is required beyond the σ_F threshold approach.
  • dft-dataset-generation: When validation reveals systematic failures (missing config_type, wrong chemistry for a species, poor surface accuracy), the remedy is always to generate more targeted DFT data. That skill covers how to design the supplementary DFT calculations.
  • dft-convergence: DFT reference calculations for phonon, EOS, and elastic constant validation must be computed at the same level of theory as the training data. If a discrepancy appears between MLP and DFT property references, verify that the reference DFT settings are identical to those used for training before concluding the MLP is wrong.
  • vasp-workflow: DFT reference calculations for EOS, phonon, and elastic property validation are typically run with VASP. The ISIF settings (ISIF=3 for cell relaxation, ISIF=2 for stress at fixed cell) from that skill are required for computing accurate reference stress tensors and elastic constants.
  • quantum-espresso: When QE is the reference code for validation, phonopy can use QE force outputs directly. Workflow 4's finite displacement approach applies identically; only the DFT interface changes.
  • lammps-workflow: The NVE and NVT MD stability tests in Workflow 5 can be run in LAMMPS as well as ASE. The equilibration protocol, timestep conventions, and thermostat settings from that skill apply to MACE-driven LAMMPS validation runs.
  • md-equilibration: The pre-validation equilibration of the starting structure for NVE tests must follow the protocol from that skill. Starting from a poorly equilibrated structure inflates the apparent energy drift and produces a pessimistic NVE assessment.
  • dft-phonons: The DFT reference phonon calculation that MLP phonons are validated against is described in that skill. The supercell construction, displacement protocol, and BORN charge handling for polar materials described there must match the phonopy setup in Workflow 4.
  • eos-fitting: The Birch-Murnaghan EOS fitting and the DFT reference EOS required for Workflow 3 follow the protocol in that skill. When DFT and MLP EOS are computed consistently, discrepancies in V₀ and B₀ are unambiguously attributable to the MLP.

Key References

  1. Zuo, Y., et al. (2020). Performance and cost assessment of machine learning interatomic potentials. The Journal of Physical Chemistry A, 124(4), 731–745. https://doi.org/10.1021/acs.jpca.9b08723
  2. Behler, J. (2016). Perspective: Machine learning potentials for atomistic simulations. The Journal of Chemical Physics, 145(17), 170901. https://doi.org/10.1063/1.4966192
  3. Musil, F., et al. (2021). Physics-inspired structural representations for molecules and materials. Chemical Reviews, 121(16), 9759–9815. https://doi.org/10.1021/acs.chemrev.1c00021
  4. Miksch, A. M., et al. (2021). Strategies for the construction of machine-learning potentials for accurate and efficient atomic-scale simulations. Machine Learning: Science and Technology, 2(3), 031001. https://doi.org/10.1088/2632-2153/abfd96
  5. Fu, X., et al. (2023). Forces are not enough: Benchmark and critical evaluation of machine learning force fields for molecular simulations. Transactions on Machine Learning Research. https://openreview.net/forum?id=A8pqQipwkt

示例提示词

我想在部署前系统验证这个机器学习势的弹性常数和MD稳定性。

所属插件

计算材料学:第一性原理与原子模拟面向计算材料研究:DFT 与第一性原理(VASP、Quantum ESPRESSO、CP2K、AiiDA)、缺陷/表面/声子/弹性常数、机器学习势函数训练与验证、分子动力学与增强采样、晶体性质图神经网络与生成模型。

安装该插件会一并获得同领域的 35 个 Skill / MCP。

同插件的其他科研 Skill

常见问题

机器学习势产前验证能做什么?

Covers systematic validation of ML interatomic potentials before deployment: test-set error and physical property reproduction (EOS, phonons, elastic constants). Use before trusting an MLP。

如何安装机器学习势产前验证?

在科搭 Codar 左侧点击「插件」,在「组件」页签搜索「机器学习势产前验证」并安装;在 Claude Code 或 Codex 中可以发送本页的安装提示词,或下载 Skill 包后解压到 ~/.claude/skills 或 ~/.codex/skills。

机器学习势产前验证需要哪些权限?

该科研 Skill没有声明额外权限;执行具体操作时仍由 Agent 按自身规则请求确认。

机器学习势产前验证经过官方审核吗?

是。SpeedSkills 收录的每个版本都会记录内容摘要与权限声明,当前版本已通过安全审核。