科研 Skill

机器学习势微调迁移训练

fine-tuning-mlp

Covers fine-tuning pretrained universal ML interatomic potentials on system-specific DFT data. Use when adapting a universal MLP to a new system.

发布者 Codar安全审核通过v0.1.0更新于 2026-11-18

SKILL.md

Fine-Tuning Machine-Learned Interatomic Potentials

Description

This skill covers fine-tuning pretrained universal machine-learned interatomic potentials on system-specific DFT data: when fine-tuning is the right strategy, how to prepare a small targeted dataset, how to configure MACE's --foundation_model workflow, how to handle reference energies and label offsets, how to avoid catastrophic forgetting, and how to validate and deploy the resulting model. Invoke this skill after universal-potentials benchmarking has confirmed that a pretrained model (MACE-MP-0, CHGNet, SevenNet, ORB) falls short of the accuracy target for a specific chemistry or structural regime, and before committing to full system-specific training from scratch.

Domain Context

Transfer learning for MLPs exploits a pretrained model's learned representation of atomic environments — built from millions of DFT calculations across the periodic table — as a starting point for a new, narrowly defined task. The key physical insight is that local chemical environments for a new target system (e.g., a W-Re alloy, a ZnO surface, an organic solvent at 400 K) are structurally similar to environments the pretrained model has already seen in the training database. Fine-tuning adjusts the model weights to improve accuracy within the target domain while ideally retaining the broad coverage of the pretrained backbone.

Why fine-tuning works. A universal MACE-MP-0 model trained on ~150,000 Materials Project structures has learned to embed atomic environments in a high-dimensional space that already separates coordination geometries, element types, and bonding patterns. The final readout layers (energy and force heads) map these embeddings to energies and forces. For a new target system with similar chemistry, only the readout layers and possibly the last interaction block need updating. This is analogous to transfer learning in computer vision: the early convolutional filters (low-level feature detectors) transfer; the classification head does not.

When the analogy breaks down. The transfer analogy fails when the target system has structural features genuinely absent from the pretrained model's training distribution: exotic bonding (e.g., very short bonds in high-pressure phases), elements underrepresented in the Materials Project (actinides, many rare earths), coordination numbers outside the typical range, or strong long-range interactions (charge transfer, magnetic coupling) that the local potential architecture cannot represent regardless of training data. In these cases, fine-tuning may improve accuracy modestly but cannot close the fundamental accuracy gap caused by architectural limitations or training distribution mismatch.

Catastrophic forgetting. Neural network fine-tuning on a small dataset tends to overwrite the weights that encode the pretrained model's broad knowledge, replacing them with weights that fit only the fine-tuning data. The result is a model that is accurate within the fine-tuning domain but has dramatically degraded accuracy outside it. For MLPs, this manifests as excellent force errors on the fine-tuning test set but unstable MD for configurations that drift slightly outside the fine-tuning distribution — the worst failure mode because it is not caught by static validation metrics.

Reference energy consistency. Pretrained models embed a specific reference energy convention: MACE-MP-0 uses PBE+U isolated atom reference energies computed with specific PAW pseudopotentials. Fine-tuning DFT data must use the same DFT level (functional, pseudopotential family, cutoff energy, k-mesh density) as the original training data, or the model will be trained on a mixture of reference levels that produces internally inconsistent energy predictions. A 50 meV/atom systematic error from reference-energy mismatch is invisible in force validation but corrupts all thermodynamic predictions.

MACE fine-tuning mechanism. MACE provides a --foundation_model flag that loads pretrained weights and continues training from that checkpoint. Three training modes are available: (1) full fine-tuning — all weights updated; (2) head-only fine-tuning — only the energy/force readout layers updated, all interaction blocks frozen; (3) partial fine-tuning — last N interaction blocks + readout updated, earlier blocks frozen. Head-only fine-tuning is the safest choice for very small datasets (< 100 structures) and minimizes forgetting. Full fine-tuning with a small learning rate and a mixed dataset (fine-tuning data + a fraction of the original training data) is preferred for larger datasets (500+ structures). [EXPERT REVIEW NEEDED: optimal strategy depends on model size and dataset size; these are heuristics, not guarantees]

When to Use This Skill

  • Universal potential benchmarking (universal-potentials Workflow 1) showed force MAE of 1.5–10× the target accuracy, indicating the pretrained model is close but not sufficient.
  • The target system is a chemical variant of something well-represented in the Materials Project: an alloy with a new composition, a surface of a known bulk, a polymorph of a known material, a molecular liquid with a known solute.
  • The DFT budget is limited (< 500 new DFT calculations) and training from scratch would require > 1,000 well-converged structures to reach the target accuracy.
  • The target application is screening or exploratory MD where modest improvement over the universal potential baseline is sufficient, rather than precise thermodynamic integration.
  • A magnetic material where CHGNet is used as the foundation model and improved spin-resolved force accuracy is needed for a specific composition.
  • An organic system where MACE-OFF23 is the foundation model and system-specific conformational energetics need refinement.

When Not to Use This Skill

  • When the universal potential shows no correlation with DFT forces (force MAE > 10× target or Pearson r < 0.5). The pretrained representation is not a useful starting point; training from scratch with targeted DFT data will converge faster. Use mace-training directly.
  • When the target system contains elements not supported by the foundation model. Loading weights that have no training signal for a new element's embedding will not transfer useful representations. Adding new elements requires either training from scratch or using a foundation model that was trained with those elements.
  • When the target DFT level differs from the pretrained model's level and the difference cannot be corrected by a reference energy offset. Fine-tuning MACE-MP-0 (PBE+U) on HSE06 or meta-GGA data produces a hybrid-functional model; the energy scale differences and exchange-correlation contributions will be absorbed inconsistently into the model weights. Train a new model at the target DFT level if functional consistency is required.
  • When the fine-tuning dataset has fewer than ~50 structures. With very few structures, there is no meaningful train/validation split; overfitting is almost guaranteed. Expand the dataset first via active-learning-mlp or targeted DFT (dft-dataset-generation), then fine-tune.
  • When long-range electrostatics or explicit charge transfer dominate the target property. Fine-tuning a local cutoff model on ionic supercells, charged defects, or strongly polar surfaces does not add long-range physics to the architecture; it will fit the training set but fail at distances beyond the cutoff. A model with explicit long-range treatment is required.
  • When the fine-tuned model will be used as a "DFT label source" for a second model. A fine-tuned MLP's predictions are still MLP predictions, not DFT labels. Using fine-tuned model energies as training targets for another model compounds approximation errors.

Core Concepts

  • Foundation model: The pretrained universal MLP used as the starting point for fine-tuning. In MACE, loaded via --foundation_model=<path-or-model-name>. The foundation model's architecture (cutoff radius, number of interactions, hidden irreps, correlation order, max_ell) is inherited; hyperparameters cannot be changed without retraining from scratch.
  • Head-only fine-tuning: Only the final energy-readout and force-readout linear layers are updated; all message-passing interaction blocks are frozen. Fastest convergence, least catastrophic forgetting, but limited accuracy improvement — can only rescale and shift the energy landscape, not reshape local environment sensitivity. Appropriate when the fine-tuning dataset is very small (50–200 structures).
  • Full fine-tuning: All model weights are updated from the pretrained initialization. Produces the best accuracy on the fine-tuning domain if the dataset is large enough (500+ structures) but risks catastrophic forgetting if the learning rate is too large or the dataset is too small.
  • Mixed-dataset training: Including a fraction of the original training data (or a proxy for it) alongside the fine-tuning data. Prevents catastrophic forgetting by keeping the loss on the broad distribution non-zero. Requires access to the original training dataset or a representative subset, which is not always available for commercial or closed-source foundation models. [EXPERT REVIEW NEEDED]
  • Learning rate schedule for fine-tuning: Start with a learning rate 10–100× lower than the from-scratch training rate. For MACE from-scratch, LR ≈ 0.01; for fine-tuning, LR ≈ 1e-3 to 1e-4 depending on the fine-tuning dataset size and the number of frozen layers. A warm restart schedule (cosine annealing) or plateau-based LR reduction is preferred over a fixed schedule. [EXPERT REVIEW NEEDED]
  • Reference energy offset (ΔE0): The difference between the DFT isolated atom energies used for fine-tuning and those used for the foundation model's training. If the fine-tuning DFT uses different pseudopotentials, a different cutoff energy, or a different DFT functional, the absolute energies will differ by a constant per-species offset. This offset can be subtracted before training if the DFT levels are otherwise comparable; if they differ fundamentally, fine-tuning should not be used.
  • Label consistency: All fine-tuning structures must be labeled at the same DFT level: same functional, same pseudopotential version, same k-mesh density protocol, same convergence criteria, same POTCAR files (for VASP). A mixed-label dataset produces systematic energy errors that inflate validation metrics without producing a physically meaningful model.
  • Domain coverage of fine-tuning data: The fine-tuning dataset must sample the configurations the model will encounter in production: the correct temperature range, relevant structural distortions (vacancies, surface terminations, liquid configurations), and the relevant composition range. A fine-tuned model accurately validated on equilibrium structures at 0 K may fail in 1000 K MD if the fine-tuning data did not include high-temperature configurations.
  • Stiffness parameter / regularization: In some fine-tuning frameworks, a regularization term encourages the fine-tuned weights to stay close to the pretrained weights, controlling forgetting explicitly. MACE does not implement this natively; it is approximated by using a low LR and a short training run. [EXPERT REVIEW NEEDED]
  • Validation split for fine-tuning: Even with a small fine-tuning dataset, a held-out validation split (at minimum 10–20 structures) is required to detect overfitting. Cross-validation is preferred for datasets < 200 structures. The validation set must be drawn from the same structural distribution as the training set (same temperature, same composition range) but must not overlap with it.
  • MACE stage-one vs. stage-two training: MACE training proceeds in two stages; --foundation_model starts from the pretrained stage-two checkpoint. Fine-tuning runs only stage two. The stage-one checkpoint from the foundation model is not reused.

Key Workflows

Full steps for each part are in references/key-workflows/; open the one you need:

Common Failure Modes

FailureSymptomLikely CauseDiagnosticFix
Overfitting to a small fine-tuning setValidation force MAE is very low (< 20 meV/Å) but MD at target temperature diverges within a few ps; test set error is good but production behavior is poorFine-tuning dataset does not cover the configurations visited in production MD; model has memorized the training structuresRun audit_finetune_dataset_coverage on a pre-equilibration trajectory; compute SOAP dissimilarity between production frames and fine-tuning datasetExpand fine-tuning dataset to cover the target temperature and structural distortions (Workflow 4); run active learning after fine-tuning to fill gaps
Catastrophic forgettingFine-tuned model is accurate on the target domain but has force MAE 3–10× higher than the foundation model on OOD structures; model is unusable outside the fine-tuning composition or temperatureLearning rate too high or too many epochs; fine-tuning dataset dominates the loss and overwrites the pretrained representation; no anti-forgetting strategyRun forgetting_and_improvement_report with an OOD test set; compare OOD errors before and after fine-tuningSwitch to head-only or partial regime; reduce LR by 5–10×; add foundation data to the training mix (add_foundation_fraction ≥ 0.3); reduce max epochs
Reference energy mismatchPredicted formation energies are systematically shifted by 50–500 meV/atom even when forces are accurate; convex hull stability ordering is wrongFine-tuning DFT uses different pseudopotentials, cutoff energy, or functional than the foundation model's training data; delta_E0 is large for one or more speciesRun compute_reference_energy_offsets; ifΔE0
Label inconsistency across fine-tuning iterationsCommittee σ_F is elevated even for equilibrium structures; model errors are larger than expected from the fine-tuning dataset sizeFine-tuning DFT was run across multiple iterations with different INCAR settings, ENCUT, or k-mesh; some structures labeled at tighter convergence, others at looserRun check_label_consistency; compare INCAR hashes across all fine-tuning calculationsRe-run all fine-tuning calculations with a frozen, canonical INCAR; do not mix structures from different convergence campaigns in the same fine-tuning set
False improvement from data leakageFine-tuned model shows excellent test-set accuracy but production accuracy is worse than expectedTest set structures were similar or identical to training structures (same configuration type, same temperature, same supercell size); the model is interpolating within the training manifoldVerify train/test split with SOAP dissimilarity check; ensure test structures were not used for any hyperparameter or checkpoint selectionUse prototype-aware or temperature-stratified splitting; ensure test set contains structures from different MD frames, different supercell sizes, or different distortion amplitudes
Ensemble collapse after fine-tuningFine-tuned committee σ_F is uniformly < 2 meV/Å even for thermally distorted structures; active learning queries all structures or noneFine-tuning on a homogeneous small dataset drove all committee members to the same solution; seed diversity is insufficient when the dataset is the dominant constraintRun verify_finetune_did_not_destroy_committee_diversity; if mean pairwise MAE < 3 meV/Å, collapse has occurredSwitch to head-only fine-tuning which maintains more diversity; supplement with SOAP dissimilarity OOD detection as a backup to committee σ_F
Fine-tuning outside pretrained model supportFine-tuning on structures containing elements not in the foundation model's vocabulary; training appears to run but energy predictions for those elements are randomFoundation model has no trained embedding for the new element; the MACE element table is fixed at initializationCheck foundation model element coverage list before fine-tuning; test with a single-element supercell of the new elementTrain a new model from scratch that includes the new element from the beginning; do not attempt to add new elements via fine-tuning
Unstable MD after fine-tuning with improved static errorsFine-tuned model passes static force/energy validation but NVE MD drifts rapidly (> 1 meV/atom/ps) or diverges at target temperatureFine-tuning improved accuracy for near-equilibrium structures while degrading the model's behavior for rare, high-energy configurations that arise during MDRun NVE stability test (Workflow 5 in mlp-validation) on the fine-tuned model at the target temperature; compare energy drift vs. foundation modelInclude high-energy configurations from MD (displacement > 0.15 Å) in the fine-tuning dataset; add random displacement structures at the target temperature amplitude

Best Practices

  • Always benchmark the universal potential before deciding to fine-tune. The universal-potentials Workflow 1 benchmark takes < 1 hour and quantifies whether fine-tuning is warranted. If the universal potential is already accurate enough, fine-tuning adds risk (forgetting, label inconsistency) with no benefit.
  • Start with head-only fine-tuning on small datasets. For fewer than 200 fine-tuning structures, head-only is the safest strategy: it cannot forget the pretrained representation. Evaluate whether head-only improvement is sufficient before escalating to partial or full fine-tuning.
  • Always supply an OOD test set for the forgetting check. The most common oversight in fine-tuning evaluations is skipping the OOD check. Forgetting is silent in in-domain metrics. A 20-structure OOD test set sampled from the Materials Project for the same elements takes minutes to evaluate and can detect catastrophic forgetting before deployment.
  • Use explicit E0s when the fine-tuning dataset is small or compositionally narrow. The --E0s="average" option regresses reference energies from the training dataset. With fewer than 100 structures of a single composition, the regression is underdetermined and produces spurious E0 values. Provide pre-computed isolated atom energies instead.
  • Record the foundation model SHA-256 alongside every fine-tuning result. Universal potential weights change between package releases. A fine-tuning run that used mace_mp(model="medium") from package version 0.3.5 will not be reproducible with version 0.3.7 unless the specific checkpoint file is archived. Always download and record the exact checkpoint.
  • Run the full mlp-validation protocol on the fine-tuned model, not just static force errors. Static force MAE on the fine-tuning test set is insufficient to certify deployment readiness. NVE stability, phonon frequencies, and the NVT ensemble test (from mlp-validation) must all be run on the fine-tuned model before production use.
  • Use add_foundation_fraction ≥ 0.3 for full fine-tuning regimes when anti-forgetting is important. Adding 30–50% of the fine-tuning training set size in foundation structures to the training batch significantly reduces forgetting with modest training cost increase. This is especially important when the fine-tuned model will be used both within and outside the target domain.
  • Do not use the fine-tuned model's predictions as labels for a second model. A fine-tuned MLP predicts at MLP accuracy, not DFT accuracy. Using its predictions as labels for training another model compounds errors. All labels must come from actual DFT calculations.

Integration with Other Skills

Moved to references/integration-with-other-skills.md to keep this file short; read it when this part is needed.

Key References

  1. Batatia, I., et al. (2023). A foundation model for atomistic materials chemistry. arXiv:2401.00096. https://arxiv.org/abs/2401.00096
  2. Deng, B., et al. (2023). CHGNet as a pretrained universal neural network potential for charge-informed atomistic modelling. Nature Machine Intelligence, 5, 1031–1041. https://doi.org/10.1038/s42256-023-00716-3
  3. Goodfellow, I., et al. (2013). An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv:1312.6211. https://arxiv.org/abs/1312.6211
  4. Zaverkin, V., et al. (2023). Transfer learning for chemically accurate interatomic neural network potentials. Physical Chemistry Chemical Physics, 25, 5383. https://doi.org/10.1039/D2CP05793J
  5. Kovács, D. P., et al. (2023). MACE-OFF23: Transferable machine learning force fields for organic molecules. arXiv:2312.15211. https://arxiv.org/abs/2312.15211

示例提示词

帮我用一小批DFT数据对MACE基础模型做体系专属的微调训练。

所属插件

计算材料学:第一性原理与原子模拟面向计算材料研究:DFT 与第一性原理(VASP、Quantum ESPRESSO、CP2K、AiiDA)、缺陷/表面/声子/弹性常数、机器学习势函数训练与验证、分子动力学与增强采样、晶体性质图神经网络与生成模型。

安装该插件会一并获得同领域的 35 个 Skill / MCP。

同插件的其他科研 Skill

常见问题

机器学习势微调迁移训练能做什么?

Covers fine-tuning pretrained universal ML interatomic potentials on system-specific DFT data. Use when adapting a universal MLP to a new system。

如何安装机器学习势微调迁移训练?

在科搭 Codar 左侧点击「插件」,在「组件」页签搜索「机器学习势微调迁移训练」并安装;在 Claude Code 或 Codex 中可以发送本页的安装提示词,或下载 Skill 包后解压到 ~/.claude/skills 或 ~/.codex/skills。

机器学习势微调迁移训练需要哪些权限?

该科研 Skill没有声明额外权限;执行具体操作时仍由 Agent 按自身规则请求确认。

机器学习势微调迁移训练经过官方审核吗?

是。SpeedSkills 收录的每个版本都会记录内容摘要与权限声明,当前版本已通过安全审核。