科研 Skill
机器学习势不确定性量化
uncertainty-mlp
Covers practical uncertainty quantification for ML interatomic potentials: committee/ensemble models, model-deviation computation. Use when estimating confidence in MLP predictions.
SKILL.md
Uncertainty Quantification for ML Interatomic Potentials
Description
This skill covers practical uncertainty quantification (UQ) for machine-learned interatomic potentials: committee (ensemble) models, model-deviation computation for forces, energies, and stresses, calibration of uncertainty thresholds against DFT errors, SOAP-based structural dissimilarity as a complementary out-of-distribution (OOD) detector, conformal prediction for rigorous confidence intervals, uncertainty propagation through MD trajectories, and stopping/extrapolation criteria. Invoke this skill when designing the active learning acquisition strategy for a new MLP campaign, when diagnosing silent extrapolation in production MD, or when quantifying the reliability of MLP predictions for properties that depend on the model's generalization behavior.
Domain Context
An MLP evaluated on a structure outside its training distribution will produce incorrect forces with no error message and no indication of failure. The only way to detect this before a catastrophic MD trajectory is through an uncertainty estimate that flags the structure as novel relative to the training set. UQ for MLPs is therefore a safety mechanism, not a statistical nicety.
Epistemic versus aleatoric uncertainty. Epistemic uncertainty arises from insufficient training data: the model has not seen enough structures to constrain its predictions in a region of configuration space. This type of uncertainty can be reduced by adding more training data — it is the signal that drives active learning. Aleatoric uncertainty arises from irreducible noise in the labels (DFT numerical noise, finite k-mesh errors, SCF threshold effects). For well-converged DFT labels (EDIFF=1e-6, PREC=Accurate), aleatoric uncertainty is typically < 1 meV/Å for forces and is negligible relative to model extrapolation errors. In practice, committee model disagreement primarily measures epistemic uncertainty, but cannot cleanly separate it from aleatoric contributions.
The committee model approach. The dominant practical UQ method for neural network potentials is the ensemble (committee): N independently trained models with identical architecture but different random seeds or different training data subsets. The standard deviation of their force predictions at a given structure, σ_F, is the uncertainty proxy. Structures with high σ_F are candidates for DFT labeling in the active learning loop. Structures with catastrophically high σ_F are likely extrapolation cases where at least one committee member has produced garbage forces — these should be inspected before being sent to DFT.
What committee models do not detect. The committee approach has a fundamental blind spot: if all committee members agree on the same wrong answer, σ_F is low and the structure passes the confidence filter. This happens in two well-known failure modes. First, if the committee was trained on the same data with only different random seeds, the members are all trained to the same DFT reference surface and will agree on any structure that is similar to the training distribution but whose correct DFT energy is systematically different from what the training surface predicts (e.g., a structure with a subtle structural distortion not covered by training). Second, for universal potentials (MACE-MP-0, CHGNet), a "committee" of fine-tuned versions of the same foundation model will share the same base representation and will not disagree enough in regions near the fine-tuning data. In both cases, SOAP-based structural dissimilarity is a complementary check that can flag structures as novel even when the committee is falsely confident.
Calibration. A committee model that reports σ_F = 0.10 eV/Å is only useful if that number is interpretable: does it mean the true force error is likely < 0.10 eV/Å, or < 1.0 eV/Å? Calibration maps σ_F (or any other uncertainty proxy) to actual DFT force errors on a held-out validation set. A calibrated uncertainty estimate satisfies: the true error is below the predicted bound with the stated probability. Uncalibrated thresholds (set by intuition rather than by reference to DFT errors on the same system) routinely cause either excessive DFT queries (σ_low set too low) or silent extrapolation (σ_high set too high).
Uncertainty in production MD. A structure visited in production MD may have any uncertainty level. Structures with σ_F > σ_high should halt or restart the MD trajectory; structures with σ_low < σ_F < σ_high are candidates for active learning; structures with σ_F < σ_low are confident interpolation. Continuously monitoring σ_F during MD adds computational overhead (each step requires N force evaluations instead of 1), which is why fast committee evaluation and selective monitoring strategies are important in practice. [EXPERT REVIEW NEEDED: overhead tolerance depends on system size and committee size]
When to Use This Skill
- Designing the active learning query strategy: computing σ_F on exploration MD frames to select which structures to label with DFT.
- Diagnosing whether a production MD trajectory is extrapolating before or after the trajectory diverges.
- Deciding whether to deploy a system-specific MLP or a universal potential for a new chemical system, by checking whether the universal potential's training distribution covers the target.
- Quantifying MLP prediction reliability for a property report or paper: committee variance provides an estimate of how much the prediction would change if the training data were different.
- Detecting the onset of phase transitions or structural transformations in MD where the model begins sampling uncharted configuration space.
When Not to Use This Skill
- As a substitute for
mlp-validation. Committee uncertainty and structural dissimilarity are indicators of likely error, not measurements of actual error. The validation trinity (static test-set errors, physical property reproduction, MD stability) from that skill is required to characterize actual accuracy. A low σ_F does not certify the model is correct — it certifies the committee is consistent. - For aleatoric noise estimation. If the goal is to quantify DFT numerical uncertainty (label noise), that requires repeating DFT calculations with perturbed settings, not running a committee. Committee variance is not a reliable estimate of DFT label noise.
- For universal potentials without fine-tuning. A committee of frozen universal potential checkpoints (e.g., multiple MACE-MP-0 release versions) spans model-version uncertainty, not the uncertainty of the potential for a specific chemical system. This is useful for benchmarking but is not the same as epistemic uncertainty relative to the target DFT level.
- When the computational budget for N×inference is prohibitive. For very large systems (> 100,000 atoms) where a single MLP evaluation costs hours, running a 4-member committee is impractical. Use single-model last-layer uncertainty, latent-space distance, or SOAP dissimilarity instead. [EXPERT REVIEW NEEDED]
Core Concepts
- Committee model (deep ensemble): N independently trained MLP instances with identical architecture but different random seeds. Each member sees the same training data but converges to a different local minimum of the loss surface. The spread of their predictions measures epistemic uncertainty. Minimum committee size: N=4; recommended: N=8 for well-calibrated estimates. Training N models multiplies training cost by N.
- Model deviation (σ_F): The standard deviation of per-component force predictions across committee members at a given structure. Computed per atom as the Frobenius norm of the force standard deviation tensor. Aggregated to structure level as the maximum or mean per-atom σ_F. Typical units: eV/Å. Related but distinct from
σ_E(energy deviation) andσ_S(stress deviation). - Uncertainty thresholds (σ_low, σ_high): The two-tier classification scheme. Structures with σ_F < σ_low are well-converged (confident interpolation); the MD trajectory is continued without DFT labeling. Structures with σ_low ≤ σ_F < σ_high are candidates for active learning. Structures with σ_F ≥ σ_high are likely extrapolation or unphysical; the MD step is rejected or the structure is flagged. Thresholds must be calibrated against actual DFT errors on a validation set, not set by intuition. [EXPERT REVIEW NEEDED: typical ranges are 0.05–0.20 eV/Å for σ_low and 0.50–2.0 eV/Å for σ_high for metallic systems]
- Calibration: The process of mapping σ_F to actual DFT force errors using a held-out validation set with known DFT labels. A calibrated committee satisfies: P(|F_DFT - F̄_committee| < α · σ_F) ≥ 1 - δ, where α and δ are calibration parameters. Calibration curves plot σ_F against actual error for validation set structures.
- SOAP dissimilarity: Euclidean distance between a structure's SOAP descriptor and the nearest training set structure in SOAP space. A complementary OOD detector that is nonzero even when the committee unanimously agrees on a wrong answer. Requires a precomputed SOAP descriptor database of the training set.
- Coverage radius (δ_cover): The maximum SOAP distance from a training structure within which the MLP is expected to interpolate reliably. Estimated from the validation set: δ_cover ≈ max SOAP distance to nearest training neighbor across validation structures that pass all MLP validation checks.
- Conformal prediction: A distribution-free framework for constructing prediction intervals with rigorous marginal coverage guarantees. For a calibration set of size n, conformal intervals contain the true value with probability ≥ 1 - α on exchangeable test points. Applied to forces: fit a nonconformity score (e.g., committee σ_F normalized by magnitude), find the 1-α quantile on the calibration set, use this as the force interval half-width for test structures.
- Per-atom vs. structure-level uncertainty: Committee σ_F is computed per atom. Structure-level aggregation options: maximum (worst-case; most conservative for active learning), mean (average environment uncertainty), 90th percentile (robust to single-atom outliers). Maximum aggregation is standard for stopping/extrapolation criteria; mean is more appropriate for diversity-weighted active learning queries.
- Energy deviation (σ_E): Per-atom energy standard deviation across committee. Less sensitive to local environment changes than σ_F; useful for detecting systematic PES curvature disagreements that affect thermodynamic properties without triggering large force errors. Monitor alongside σ_F for NPT MD validation.
- Ensemble collapse: The failure mode where committee members converge to nearly identical models despite different seeds. Occurs when the training set is small and homogeneous, the loss landscape has a deep and narrow global minimum, or the optimizer is deterministic enough to overcome seed diversity. Detected by checking that committee pairwise test-set force MAE differences are > 2× the individual member's test-set noise floor.
Key Workflows
Full steps for each part are in references/key-workflows/; open the one you need:
- Workflow 1: Training a MACE committee and verifying member diversity —
references/key-workflows/01-workflow-1-training-a-mace-committee-and-verifyi.md: A properly diverse committee is the foundation of all downstream UQ. Members trained with different seeds on the same dataset will differ only if the training is genuinely stochastic and the loss landscape has multiple distinct minima. - Workflow 2: Per-frame force, energy, and stress model deviation —
references/key-workflows/02-workflow-2-per-frame-force-energy-and-stress-mod.md: Computing σ_F, σ_E, and σ_S efficiently during MD or for a batch of candidate structures. The aggregation from atom-level to structure-level is a critical design choice. - Workflow 3: Calibrating σ_F thresholds against actual DFT errors —
references/key-workflows/03-workflow-3-calibrating-f-thresholds-against-actu.md: Thresholds that are not calibrated against the actual DFT force errors on the target system are guesses. This workflow uses the held-out test set (with known DFT labels) to fit calibration parameters. - Workflow 4: SOAP-based structural dissimilarity as complementary OOD detector —
references/key-workflows/04-workflow-4-soap-based-structural-dissimilarity-a.md: Committee variance is blind to confident-but-wrong extrapolation. SOAP dissimilarity catches structures that are geometrically novel relative to the training set, even if the committee happens to agree. - Workflow 5: Conformal prediction for rigorous per-structure force intervals —
references/key-workflows/05-workflow-5-conformal-prediction-for-rigorous-per.md: Conformal prediction produces intervals with guaranteed marginal coverage without distributional assumptions. Applicable to force prediction as a post-hoc wrapper around any MLP. - Workflow 6: Uncertainty monitoring in LAMMPS trajectories and stopping criteria —
references/key-workflows/06-workflow-6-uncertainty-monitoring-in-lammps-traj.md: Post-processing a LAMMPS trajectory through the committee to identify extrapolation onset, without running the committee inline during MD.
Common Failure Modes
Moved to references/common-failure-modes.md to keep this file short; read it when this part is needed.
Best Practices
- Train at least 4 committee members, preferably 8. With N=2 or N=3, the standard deviation has high statistical noise (std of 2 values is poorly estimated). N=8 provides a reliable estimate at 8× training cost. For large models where 8× is unaffordable, use N=4 and increase the seed spacing.
- Verify committee diversity immediately after training with
verify_committee_diversity. A committee where members agree within 3 meV/Å on a diverse test set is collapsed and will not provide useful uncertainty estimates. Do not proceed to active learning with a collapsed committee. - Calibrate σ_low and σ_high on every new chemical system using Workflow 3. Thresholds calibrated for a W-Ta alloy at 1000 K will not transfer to an oxide at 500 K or an organic molecule. The calibration takes < 1 hour on a pre-existing test set and prevents both excessive DFT spending and silent extrapolation.
- Always supplement committee σ_F with SOAP dissimilarity for critical production runs. The two signals are complementary: σ_F is sensitive to epistemic uncertainty within the training distribution; SOAP dissimilarity is sensitive to structural novelty regardless of committee agreement. Use both in parallel and flag structures where they disagree (Workflow 4).
- Use per-atom σ_F for active learning queries, not structure-level max σ_F. Selecting structures where a single atom drives up σ_F_max often targets high-force short-contact events that are edge cases, not the most informative structures. Filter by normalized (relative) σ_F or by mean σ_F before FPS diversity selection.
- Record the committee manifest (model paths, SHA-256 hashes, seeds, and calibration parameters) in the project log. A committee evaluated with a different model file than what was calibrated against will have different σ_F values and different threshold semantics. See
reproducibility-workflowfor how to track this. - Run the NVE energy conservation test from
mlp-validationbefore each new AL iteration's model. A model with poor NVE stability will produce diverging trajectories before σ_F rises above σ_high; NVE drift is an independent check that the model is physically stable in the well-sampled regime. - Do not use committee uncertainty as the sole stopping criterion for active learning. The fraction of frames with σ_F > σ_low dropping below 1% is a necessary but not sufficient stopping criterion. Always run the full
mlp-validationprotocol on the final model before declaring convergence.
Integration with Other Skills
- mace-training: Committee members are trained using the same hyperparameters as the single-model workflow described there. The
--seedargument,--ema,--scaling, and--default_dtypeflags all apply identically to committee training. The best-practice note there to record the model checksum extends to every committee member. - active-learning-mlp: Committee σ_F is the primary uncertainty signal in that skill's Workflow 2 and Workflow 3. The σ_low, σ_high, and σ_query thresholds defined by calibration in Workflow 3 above map directly to the
SIGMA_LOW,SIGMA_HIGH, andSIGMA_QUERYconstants in that skill. The FPS diversity selection in that skill's Workflow 3 operates on the candidate pool filtered by this skill's uncertainty classification. - mlp-validation: Static test-set error analysis (Workflow 1 of that skill) provides the ground truth against which committee σ_F is calibrated (Workflow 3 above). The held-out test set used for static validation must also be the calibration set for uncertainty thresholds. The
check_extrapolation_coveragefunction in that skill uses committee σ_F with the thresholds produced here. - mlp-to-lammps: Post-processing LAMMPS trajectories through the Python committee (Workflow 6 above) uses structures extracted from LAMMPS dump files. The committee force evaluation is done in Python/ASE after the LAMMPS run; LAMMPS itself uses only a single model for production. The model SHA-256 from the deployment validation in that skill must match one of the committee member checksums.
- universal-potentials: The limitations of committee UQ for fine-tuned universal potentials are described in the Domain Context and in the failure mode table. That skill covers when universal potentials are appropriate without fine-tuning; the SOAP dissimilarity approach described here is the recommended OOD check for any universal potential deployment.
- fine-tuning-mlp: When fine-tuning a universal potential on system-specific DFT data, the committee for UQ should be built from fine-tuned members, not from the foundation model's checkpoints. The SOAP dissimilarity coverage radius must be fitted against the fine-tuning training set, not the pre-training set.
- dft-dataset-generation: The structures flagged for DFT labeling by committee uncertainty are the inputs to that skill's VASP and QE labeling pipelines. The settings hash from the canonical INCAR must be consistent across all iterations; the
DatasetManifesthash check prevents accidental label inconsistency between committee members trained on data from different iterations. - dft-convergence: Inconsistent DFT settings between committee training iterations is a silent failure mode that elevates committee σ_F artifactually. The settings hash established by that skill's convergence protocol must be frozen before the first AL iteration and checked in every subsequent oracle call.
- lammps-workflow: Exploration MD trajectories used for active learning are run via LAMMPS. The dump frequency and dump format settings from that skill determine how many frames are available for committee post-processing. A dump frequency that is too low (every 1000 steps) may miss short-lived high-uncertainty configurations.
- md-equilibration: Committee σ_F during the equilibration phase provides an early warning that the MLP is extrapolating before the full production run is committed. The NVE energy conservation test from that skill and committee σ_F are complementary: NVE tests stability within the sampled configurations; σ_F flags the onset of novel configurations.
- reproducibility-workflow: The committee manifest (model paths, SHA-256, seeds, calibration parameters σ_low/σ_high, calibration factor α, calibration set checksum) is the per-campaign provenance record for the UQ pipeline. It belongs alongside the dataset manifest and the MLflow run records described in that skill.
- materials-featurization: The SOAP descriptor parameters used for structural dissimilarity in Workflow 4 must be consistent with those used for FPS diversity filtering in
active-learning-mlpanddft-dataset-generation. That skill'ssave_feature_metadatapattern should be applied to the SOAP parameters used inSOAPDissimilarityDetectorto ensure the same descriptor is used at every stage.
Key References
- Schran, C., et al. (2020). Committee neural network potentials control generalization errors and enable active learning. The Journal of Chemical Physics, 153(10), 104105. https://doi.org/10.1063/5.0016004
- Imbalzano, G., et al. (2021). Uncertainty estimation for molecular dynamics and sampling. The Journal of Chemical Physics, 154(7), 074102. https://doi.org/10.1063/5.0036522
- Vandermause, J., et al. (2020). On-the-fly active learning of interpretable Bayesian force fields for atomistic rare events. npj Computational Materials, 6, 20. https://doi.org/10.1038/s41524-020-0283-z
- Batatia, I., et al. (2022). MACE: Higher order equivariant message passing neural networks for fast and accurate force fields. Advances in Neural Information Processing Systems, 35. https://arxiv.org/abs/2206.07697
- Angelopoulos, A. N., & Bates, S. (2023). Conformal prediction: A gentle introduction. Foundations and Trends in Machine Learning, 16(4), 494–591. https://doi.org/10.1561/2200000101
示例提示词
帮我用委员会模型估计这个机器学习势预测力的不确定性并设定阈值。
所属插件
安装该插件会一并获得同领域的 35 个 Skill / MCP。
同插件的其他科研 Skill
- 主动学习构建机器学习势训练集Runs active-learning workflows for ML interatomic potentials: uncertainty-guided MD exploration, DFT oracle labeling, query strategy. Use when expanding an MLP training set.
- AiiDA计算工作流管理Covers AiiDA, a provenance-first Python workflow engine for computational materials science: calculation input generation, remote submission. Use when orchestrating reproducible DFT/MD campaigns.
- ASE原子模拟环境Covers the Atomic Simulation Environment (ASE) as an atomistic workflow hub: Atoms objects, calculators, optimizers, trajectories. Use when scripting or converting structures across DFT/MD codes.
- 粗粒化分子动力学模拟Covers coarse-grained molecular dynamics: mapping atomistic structures to CG beads, parameterizing Martini/IBI/force-matching force fields. Use when atomistic timescales are too short.
- CP2K第一性原理分子动力学Covers end-to-end DFT, AIMD, and atomistic workflows with CP2K using the Gaussian-and-plane-waves method: input construction, convergence testing. Use for CP2K-based simulations.
- 扩散模型生成晶体结构Covers de novo crystal structure generation with score-based diffusion and flow-matching models (MatterGen, DiffCSP, CDVAE). Use when generating candidate structures, not analyzing existing ones.
- 晶体图神经网络性质预测Covers graph neural networks for crystal and periodic-material property prediction: crystal graph construction, periodic neighbor finding. Use when building GNN property predictors for crystals.
- 平面波DFT收敛性测试Covers systematic convergence testing for plane-wave DFT: cutoff energy, k-point mesh density, smearing. Use before trusting DFT energies, forces, or derived properties as converged.
- 机器学习势DFT数据集生成Covers generating DFT-labeled datasets for training ML interatomic potentials: configuration-space coverage. Use when building an MLP training dataset from scratch.
- 点缺陷第一性原理计算Covers first-principles point-defect modeling: vacancy/interstitial/antisite/substitutional construction, charge-state calculations. Use for defect thermodynamics studies.
- 第一性原理声子谱计算Covers lattice dynamics from first principles: phonon dispersion, phonon DOS. Use when computing vibrational or thermodynamic properties from DFT.
- 表面模型DFT计算Covers surface slab model construction and DFT calculation: Miller-index generation, termination selection, slab/vacuum convergence. Use for surface science and catalysis DFT studies.
- 弹性常数计算工作流Computes single-crystal and polycrystalline elastic constants from DFT or MD stress-strain data and checks mechanical stability. Use when deriving stiffness tensors or bulk/shear modulus.
- 增强采样自由能计算Covers enhanced sampling methods for atomistic MD (metadynamics, umbrella sampling. Use for rare-event or slow-transition sampling.
- 状态方程拟合Covers equation-of-state fitting from DFT/ML-potential energy-volume data: Birch-Murnaghan, Vinet, and Murnaghan forms. Use when fitting an EOS to compression data.
- 机器学习势微调迁移训练Covers fine-tuning pretrained universal ML interatomic potentials on system-specific DFT data. Use when adapting a universal MLP to a new system.
- 分子动力学自由能计算Covers quantitative free-energy calculations from MD: alchemical FEP/TI/BAR/MBAR, umbrella-sampling PMF. Use when computing thermodynamic free energies from simulation.
- GROMACS分子动力学工作流Covers end-to-end MD simulations with GROMACS: topology/.mdp construction, energy minimization, NVT/NPT equilibration. Use for GROMACS-based biomolecular or materials MD.
- 高通量DFT自动化筛选Covers automated large-scale DFT screening: workflow graphs with atomate2/jobflow/AiiDA, input-set standardization, custodian error handling. Use for high-throughput DFT screening campaigns.
- LAMMPS分子动力学工作流Covers end-to-end MD simulations with LAMMPS: input script anatomy, force-field deployment, ensemble control, equilibration. Use for LAMMPS-based classical molecular dynamics.
- MACE机器学习势训练Covers training MACE machine-learned interatomic potentials: extxyz dataset preparation, isolated-atom energies, train/val/test splitting. Use for MACE-specific MLP training workflows.
- MatGL图神经网络框架Covers building, applying, validating, and fine-tuning MatGL graph-neural-network models for property prediction and interatomic potentials. Use for pretrained GNN workflows via MatGL.
- 分子动力学轨迹后处理分析Covers quantitative post-processing of atomistic MD trajectories: loading LAMMPS/GROMACS/ASE trajectories and computing RDF, coordination numbers. Use for structural MD trajectory analysis.
- 分子动力学平衡过程设计Covers bringing an atomistic simulation from its initial configuration to a representative equilibrium state before production statistics: ensemble. Use before collecting production MD data.
常见问题
机器学习势不确定性量化能做什么?
Covers practical uncertainty quantification for ML interatomic potentials: committee/ensemble models, model-deviation computation. Use when estimating confidence in MLP predictions。
如何安装机器学习势不确定性量化?
在科搭 Codar 左侧点击「插件」,在「组件」页签搜索「机器学习势不确定性量化」并安装;在 Claude Code 或 Codex 中可以发送本页的安装提示词,或下载 Skill 包后解压到 ~/.claude/skills 或 ~/.codex/skills。
机器学习势不确定性量化需要哪些权限?
该科研 Skill没有声明额外权限;执行具体操作时仍由 Agent 按自身规则请求确认。
机器学习势不确定性量化经过官方审核吗?
是。SpeedSkills 收录的每个版本都会记录内容摘要与权限声明,当前版本已通过安全审核。