科研 Skill

机器学习势不确定性量化

uncertainty-mlp

Covers practical uncertainty quantification for ML interatomic potentials: committee/ensemble models, model-deviation computation. Use when estimating confidence in MLP predictions.

发布者 Codar安全审核通过v0.1.0更新于 2026-11-18

SKILL.md

Uncertainty Quantification for ML Interatomic Potentials

Description

This skill covers practical uncertainty quantification (UQ) for machine-learned interatomic potentials: committee (ensemble) models, model-deviation computation for forces, energies, and stresses, calibration of uncertainty thresholds against DFT errors, SOAP-based structural dissimilarity as a complementary out-of-distribution (OOD) detector, conformal prediction for rigorous confidence intervals, uncertainty propagation through MD trajectories, and stopping/extrapolation criteria. Invoke this skill when designing the active learning acquisition strategy for a new MLP campaign, when diagnosing silent extrapolation in production MD, or when quantifying the reliability of MLP predictions for properties that depend on the model's generalization behavior.

Domain Context

An MLP evaluated on a structure outside its training distribution will produce incorrect forces with no error message and no indication of failure. The only way to detect this before a catastrophic MD trajectory is through an uncertainty estimate that flags the structure as novel relative to the training set. UQ for MLPs is therefore a safety mechanism, not a statistical nicety.

Epistemic versus aleatoric uncertainty. Epistemic uncertainty arises from insufficient training data: the model has not seen enough structures to constrain its predictions in a region of configuration space. This type of uncertainty can be reduced by adding more training data — it is the signal that drives active learning. Aleatoric uncertainty arises from irreducible noise in the labels (DFT numerical noise, finite k-mesh errors, SCF threshold effects). For well-converged DFT labels (EDIFF=1e-6, PREC=Accurate), aleatoric uncertainty is typically < 1 meV/Å for forces and is negligible relative to model extrapolation errors. In practice, committee model disagreement primarily measures epistemic uncertainty, but cannot cleanly separate it from aleatoric contributions.

The committee model approach. The dominant practical UQ method for neural network potentials is the ensemble (committee): N independently trained models with identical architecture but different random seeds or different training data subsets. The standard deviation of their force predictions at a given structure, σ_F, is the uncertainty proxy. Structures with high σ_F are candidates for DFT labeling in the active learning loop. Structures with catastrophically high σ_F are likely extrapolation cases where at least one committee member has produced garbage forces — these should be inspected before being sent to DFT.

What committee models do not detect. The committee approach has a fundamental blind spot: if all committee members agree on the same wrong answer, σ_F is low and the structure passes the confidence filter. This happens in two well-known failure modes. First, if the committee was trained on the same data with only different random seeds, the members are all trained to the same DFT reference surface and will agree on any structure that is similar to the training distribution but whose correct DFT energy is systematically different from what the training surface predicts (e.g., a structure with a subtle structural distortion not covered by training). Second, for universal potentials (MACE-MP-0, CHGNet), a "committee" of fine-tuned versions of the same foundation model will share the same base representation and will not disagree enough in regions near the fine-tuning data. In both cases, SOAP-based structural dissimilarity is a complementary check that can flag structures as novel even when the committee is falsely confident.

Calibration. A committee model that reports σ_F = 0.10 eV/Å is only useful if that number is interpretable: does it mean the true force error is likely < 0.10 eV/Å, or < 1.0 eV/Å? Calibration maps σ_F (or any other uncertainty proxy) to actual DFT force errors on a held-out validation set. A calibrated uncertainty estimate satisfies: the true error is below the predicted bound with the stated probability. Uncalibrated thresholds (set by intuition rather than by reference to DFT errors on the same system) routinely cause either excessive DFT queries (σ_low set too low) or silent extrapolation (σ_high set too high).

Uncertainty in production MD. A structure visited in production MD may have any uncertainty level. Structures with σ_F > σ_high should halt or restart the MD trajectory; structures with σ_low < σ_F < σ_high are candidates for active learning; structures with σ_F < σ_low are confident interpolation. Continuously monitoring σ_F during MD adds computational overhead (each step requires N force evaluations instead of 1), which is why fast committee evaluation and selective monitoring strategies are important in practice. [EXPERT REVIEW NEEDED: overhead tolerance depends on system size and committee size]

When to Use This Skill

  • Designing the active learning query strategy: computing σ_F on exploration MD frames to select which structures to label with DFT.
  • Diagnosing whether a production MD trajectory is extrapolating before or after the trajectory diverges.
  • Deciding whether to deploy a system-specific MLP or a universal potential for a new chemical system, by checking whether the universal potential's training distribution covers the target.
  • Quantifying MLP prediction reliability for a property report or paper: committee variance provides an estimate of how much the prediction would change if the training data were different.
  • Detecting the onset of phase transitions or structural transformations in MD where the model begins sampling uncharted configuration space.

When Not to Use This Skill

  • As a substitute for mlp-validation. Committee uncertainty and structural dissimilarity are indicators of likely error, not measurements of actual error. The validation trinity (static test-set errors, physical property reproduction, MD stability) from that skill is required to characterize actual accuracy. A low σ_F does not certify the model is correct — it certifies the committee is consistent.
  • For aleatoric noise estimation. If the goal is to quantify DFT numerical uncertainty (label noise), that requires repeating DFT calculations with perturbed settings, not running a committee. Committee variance is not a reliable estimate of DFT label noise.
  • For universal potentials without fine-tuning. A committee of frozen universal potential checkpoints (e.g., multiple MACE-MP-0 release versions) spans model-version uncertainty, not the uncertainty of the potential for a specific chemical system. This is useful for benchmarking but is not the same as epistemic uncertainty relative to the target DFT level.
  • When the computational budget for N×inference is prohibitive. For very large systems (> 100,000 atoms) where a single MLP evaluation costs hours, running a 4-member committee is impractical. Use single-model last-layer uncertainty, latent-space distance, or SOAP dissimilarity instead. [EXPERT REVIEW NEEDED]

Core Concepts

  • Committee model (deep ensemble): N independently trained MLP instances with identical architecture but different random seeds. Each member sees the same training data but converges to a different local minimum of the loss surface. The spread of their predictions measures epistemic uncertainty. Minimum committee size: N=4; recommended: N=8 for well-calibrated estimates. Training N models multiplies training cost by N.
  • Model deviation (σ_F): The standard deviation of per-component force predictions across committee members at a given structure. Computed per atom as the Frobenius norm of the force standard deviation tensor. Aggregated to structure level as the maximum or mean per-atom σ_F. Typical units: eV/Å. Related but distinct from σ_E (energy deviation) and σ_S (stress deviation).
  • Uncertainty thresholds (σ_low, σ_high): The two-tier classification scheme. Structures with σ_F < σ_low are well-converged (confident interpolation); the MD trajectory is continued without DFT labeling. Structures with σ_low ≤ σ_F < σ_high are candidates for active learning. Structures with σ_F ≥ σ_high are likely extrapolation or unphysical; the MD step is rejected or the structure is flagged. Thresholds must be calibrated against actual DFT errors on a validation set, not set by intuition. [EXPERT REVIEW NEEDED: typical ranges are 0.05–0.20 eV/Å for σ_low and 0.50–2.0 eV/Å for σ_high for metallic systems]
  • Calibration: The process of mapping σ_F to actual DFT force errors using a held-out validation set with known DFT labels. A calibrated committee satisfies: P(|F_DFT - F̄_committee| < α · σ_F) ≥ 1 - δ, where α and δ are calibration parameters. Calibration curves plot σ_F against actual error for validation set structures.
  • SOAP dissimilarity: Euclidean distance between a structure's SOAP descriptor and the nearest training set structure in SOAP space. A complementary OOD detector that is nonzero even when the committee unanimously agrees on a wrong answer. Requires a precomputed SOAP descriptor database of the training set.
  • Coverage radius (δ_cover): The maximum SOAP distance from a training structure within which the MLP is expected to interpolate reliably. Estimated from the validation set: δ_cover ≈ max SOAP distance to nearest training neighbor across validation structures that pass all MLP validation checks.
  • Conformal prediction: A distribution-free framework for constructing prediction intervals with rigorous marginal coverage guarantees. For a calibration set of size n, conformal intervals contain the true value with probability ≥ 1 - α on exchangeable test points. Applied to forces: fit a nonconformity score (e.g., committee σ_F normalized by magnitude), find the 1-α quantile on the calibration set, use this as the force interval half-width for test structures.
  • Per-atom vs. structure-level uncertainty: Committee σ_F is computed per atom. Structure-level aggregation options: maximum (worst-case; most conservative for active learning), mean (average environment uncertainty), 90th percentile (robust to single-atom outliers). Maximum aggregation is standard for stopping/extrapolation criteria; mean is more appropriate for diversity-weighted active learning queries.
  • Energy deviation (σ_E): Per-atom energy standard deviation across committee. Less sensitive to local environment changes than σ_F; useful for detecting systematic PES curvature disagreements that affect thermodynamic properties without triggering large force errors. Monitor alongside σ_F for NPT MD validation.
  • Ensemble collapse: The failure mode where committee members converge to nearly identical models despite different seeds. Occurs when the training set is small and homogeneous, the loss landscape has a deep and narrow global minimum, or the optimizer is deterministic enough to overcome seed diversity. Detected by checking that committee pairwise test-set force MAE differences are > 2× the individual member's test-set noise floor.

Key Workflows

Full steps for each part are in references/key-workflows/; open the one you need:

Common Failure Modes

Moved to references/common-failure-modes.md to keep this file short; read it when this part is needed.

Best Practices

  • Train at least 4 committee members, preferably 8. With N=2 or N=3, the standard deviation has high statistical noise (std of 2 values is poorly estimated). N=8 provides a reliable estimate at 8× training cost. For large models where 8× is unaffordable, use N=4 and increase the seed spacing.
  • Verify committee diversity immediately after training with verify_committee_diversity. A committee where members agree within 3 meV/Å on a diverse test set is collapsed and will not provide useful uncertainty estimates. Do not proceed to active learning with a collapsed committee.
  • Calibrate σ_low and σ_high on every new chemical system using Workflow 3. Thresholds calibrated for a W-Ta alloy at 1000 K will not transfer to an oxide at 500 K or an organic molecule. The calibration takes < 1 hour on a pre-existing test set and prevents both excessive DFT spending and silent extrapolation.
  • Always supplement committee σ_F with SOAP dissimilarity for critical production runs. The two signals are complementary: σ_F is sensitive to epistemic uncertainty within the training distribution; SOAP dissimilarity is sensitive to structural novelty regardless of committee agreement. Use both in parallel and flag structures where they disagree (Workflow 4).
  • Use per-atom σ_F for active learning queries, not structure-level max σ_F. Selecting structures where a single atom drives up σ_F_max often targets high-force short-contact events that are edge cases, not the most informative structures. Filter by normalized (relative) σ_F or by mean σ_F before FPS diversity selection.
  • Record the committee manifest (model paths, SHA-256 hashes, seeds, and calibration parameters) in the project log. A committee evaluated with a different model file than what was calibrated against will have different σ_F values and different threshold semantics. See reproducibility-workflow for how to track this.
  • Run the NVE energy conservation test from mlp-validation before each new AL iteration's model. A model with poor NVE stability will produce diverging trajectories before σ_F rises above σ_high; NVE drift is an independent check that the model is physically stable in the well-sampled regime.
  • Do not use committee uncertainty as the sole stopping criterion for active learning. The fraction of frames with σ_F > σ_low dropping below 1% is a necessary but not sufficient stopping criterion. Always run the full mlp-validation protocol on the final model before declaring convergence.

Integration with Other Skills

  • mace-training: Committee members are trained using the same hyperparameters as the single-model workflow described there. The --seed argument, --ema, --scaling, and --default_dtype flags all apply identically to committee training. The best-practice note there to record the model checksum extends to every committee member.
  • active-learning-mlp: Committee σ_F is the primary uncertainty signal in that skill's Workflow 2 and Workflow 3. The σ_low, σ_high, and σ_query thresholds defined by calibration in Workflow 3 above map directly to the SIGMA_LOW, SIGMA_HIGH, and SIGMA_QUERY constants in that skill. The FPS diversity selection in that skill's Workflow 3 operates on the candidate pool filtered by this skill's uncertainty classification.
  • mlp-validation: Static test-set error analysis (Workflow 1 of that skill) provides the ground truth against which committee σ_F is calibrated (Workflow 3 above). The held-out test set used for static validation must also be the calibration set for uncertainty thresholds. The check_extrapolation_coverage function in that skill uses committee σ_F with the thresholds produced here.
  • mlp-to-lammps: Post-processing LAMMPS trajectories through the Python committee (Workflow 6 above) uses structures extracted from LAMMPS dump files. The committee force evaluation is done in Python/ASE after the LAMMPS run; LAMMPS itself uses only a single model for production. The model SHA-256 from the deployment validation in that skill must match one of the committee member checksums.
  • universal-potentials: The limitations of committee UQ for fine-tuned universal potentials are described in the Domain Context and in the failure mode table. That skill covers when universal potentials are appropriate without fine-tuning; the SOAP dissimilarity approach described here is the recommended OOD check for any universal potential deployment.
  • fine-tuning-mlp: When fine-tuning a universal potential on system-specific DFT data, the committee for UQ should be built from fine-tuned members, not from the foundation model's checkpoints. The SOAP dissimilarity coverage radius must be fitted against the fine-tuning training set, not the pre-training set.
  • dft-dataset-generation: The structures flagged for DFT labeling by committee uncertainty are the inputs to that skill's VASP and QE labeling pipelines. The settings hash from the canonical INCAR must be consistent across all iterations; the DatasetManifest hash check prevents accidental label inconsistency between committee members trained on data from different iterations.
  • dft-convergence: Inconsistent DFT settings between committee training iterations is a silent failure mode that elevates committee σ_F artifactually. The settings hash established by that skill's convergence protocol must be frozen before the first AL iteration and checked in every subsequent oracle call.
  • lammps-workflow: Exploration MD trajectories used for active learning are run via LAMMPS. The dump frequency and dump format settings from that skill determine how many frames are available for committee post-processing. A dump frequency that is too low (every 1000 steps) may miss short-lived high-uncertainty configurations.
  • md-equilibration: Committee σ_F during the equilibration phase provides an early warning that the MLP is extrapolating before the full production run is committed. The NVE energy conservation test from that skill and committee σ_F are complementary: NVE tests stability within the sampled configurations; σ_F flags the onset of novel configurations.
  • reproducibility-workflow: The committee manifest (model paths, SHA-256, seeds, calibration parameters σ_low/σ_high, calibration factor α, calibration set checksum) is the per-campaign provenance record for the UQ pipeline. It belongs alongside the dataset manifest and the MLflow run records described in that skill.
  • materials-featurization: The SOAP descriptor parameters used for structural dissimilarity in Workflow 4 must be consistent with those used for FPS diversity filtering in active-learning-mlp and dft-dataset-generation. That skill's save_feature_metadata pattern should be applied to the SOAP parameters used in SOAPDissimilarityDetector to ensure the same descriptor is used at every stage.

Key References

  1. Schran, C., et al. (2020). Committee neural network potentials control generalization errors and enable active learning. The Journal of Chemical Physics, 153(10), 104105. https://doi.org/10.1063/5.0016004
  2. Imbalzano, G., et al. (2021). Uncertainty estimation for molecular dynamics and sampling. The Journal of Chemical Physics, 154(7), 074102. https://doi.org/10.1063/5.0036522
  3. Vandermause, J., et al. (2020). On-the-fly active learning of interpretable Bayesian force fields for atomistic rare events. npj Computational Materials, 6, 20. https://doi.org/10.1038/s41524-020-0283-z
  4. Batatia, I., et al. (2022). MACE: Higher order equivariant message passing neural networks for fast and accurate force fields. Advances in Neural Information Processing Systems, 35. https://arxiv.org/abs/2206.07697
  5. Angelopoulos, A. N., & Bates, S. (2023). Conformal prediction: A gentle introduction. Foundations and Trends in Machine Learning, 16(4), 494–591. https://doi.org/10.1561/2200000101

示例提示词

帮我用委员会模型估计这个机器学习势预测力的不确定性并设定阈值。

所属插件

计算材料学:第一性原理与原子模拟面向计算材料研究:DFT 与第一性原理(VASP、Quantum ESPRESSO、CP2K、AiiDA)、缺陷/表面/声子/弹性常数、机器学习势函数训练与验证、分子动力学与增强采样、晶体性质图神经网络与生成模型。

安装该插件会一并获得同领域的 35 个 Skill / MCP。

同插件的其他科研 Skill

常见问题

机器学习势不确定性量化能做什么?

Covers practical uncertainty quantification for ML interatomic potentials: committee/ensemble models, model-deviation computation. Use when estimating confidence in MLP predictions。

如何安装机器学习势不确定性量化?

在科搭 Codar 左侧点击「插件」,在「组件」页签搜索「机器学习势不确定性量化」并安装;在 Claude Code 或 Codex 中可以发送本页的安装提示词,或下载 Skill 包后解压到 ~/.claude/skills 或 ~/.codex/skills。

机器学习势不确定性量化需要哪些权限?

该科研 Skill没有声明额外权限;执行具体操作时仍由 Agent 按自身规则请求确认。

机器学习势不确定性量化经过官方审核吗?

是。SpeedSkills 收录的每个版本都会记录内容摘要与权限声明,当前版本已通过安全审核。