科研 Skill

高通量DFT自动化筛选

high-throughput-dft

Covers automated large-scale DFT screening: workflow graphs with atomate2/jobflow/AiiDA, input-set standardization, custodian error handling. Use for high-throughput DFT screening campaigns.

发布者 Codar安全审核通过v0.1.0更新于 2026-11-18

SKILL.md

High-Throughput DFT Workflows

Description

This skill covers automated, large-scale DFT screening workflows: workflow graph construction and provenance tracking with atomate2/jobflow or AiiDA, input-set standardization, error handling with custodian, HPC job submission and restart logic, MongoDB-backed results storage, deduplication, and quality control. It connects high-throughput DFT to materials databases, ML training datasets, active learning loops, and surrogate screening campaigns. Invoke this skill when running hundreds to hundreds of thousands of DFT calculations systematically, automating a multi-step DFT pipeline (relax → static → DOS/bands → phonon → defect), or coupling DFT calculations to ML potential training or property screening.

Domain Context

High-throughput DFT (HT-DFT) is DFT run at scale: the same code, functional, and input protocol applied uniformly to a large structure library so that computed properties are mutually comparable. The value is not any single calculation but the internally consistent dataset that enables thermodynamic screening, property prediction, ML training, and active learning. This internal consistency is fragile — a single inconsistent INCAR, a different POTCAR version, or a missing convergence check can silently corrupt the comparative validity of hundreds of results.

Workflow orchestration: A DFT workflow is a directed acyclic graph (DAG) of calculations where each node is a DFT job and each edge is a data dependency. Orchestration tools (atomate2 + jobflow, FireWorks, AiiDA) manage job submission, output parsing, provenance recording, error detection, and chaining. The key distinction is between workflow definition (the graph and input sets) and workflow execution (the compute infrastructure that runs jobs). Separating these allows the same workflow graph to run on different HPC systems.

Input-set standardization: All DFT calculations in a screening campaign must use identical or explicitly documented input sets. The pymatgen InputSet machinery (MPRelaxSet, MPStaticSet, MPScanRelaxSet, MPHSEBSSet, etc.) encodes Materials Project–compatible calculation parameters, including POTCAR selection, INCAR defaults, k-mesh generation, and structure transformation. Using these sets ensures that computed energies are compatible with MP compatibility corrections and the MP thermodynamic database.

Custodian error handling: DFT calculations fail in stereotyped ways: BRMIX errors, ZPOTRF errors, EDDDAV convergence failures, electronic SCF divergence, wall-time expiry, memory errors. Custodian wraps each DFT job with error-detection handlers that parse stderr and OUTCAR, identify the failure mode, apply a documented fix (e.g., increase NCORE, lower POTIM, restart from WAVECAR), and resubmit. Every applied fix is logged with reason and parameters, forming part of the provenance record.

Provenance and reproducibility: An HT-DFT result is only as trustworthy as its provenance chain. At minimum, each result record must store: structure (with standardized cell), code version, POTCAR SHA or title line, functional and correction scheme, INCAR verbatim, k-mesh, convergence flags, custodian fixes applied, and a unique job UUID. Without this, a dataset cannot be reproduced, audited, or correctly cited.

Convergence tiers: HT-DFT workflows typically use two or three convergence tiers:

  • Screening tier: Coarser k-mesh, lower ENCUT (e.g., 1.3× ENMAX), EDIFF=1e-5, EDIFFG=-0.05. Fast; suitable for relative ranking, phase stability screening, and candidate filtering. Results are NOT comparable to experimental values and should not be published as standalone numbers.
  • Standard tier: MP-standard settings (ENCUT≥520 eV, EDIFF=1e-6, k-mesh ≥ 1000 k·atoms). Suitable for formation energies, band gaps, and inclusion in databases. The Materials Project uses this tier.
  • High-accuracy tier: ENCUT≥600 eV, EDIFF=1e-8, EDIFFG=-0.01, dense k-mesh, LREAL=False. Required for phonons, defect energies, surface energies, and training data for high-accuracy MLPs.

The most common catastrophic mistake in HT-DFT is treating screening-tier results as publication-quality data. [EXPERT REVIEW NEEDED: the exact ENCUT and k-mesh thresholds that define "screening" vs "standard" are institution-dependent; always document which tier was used]

MongoDB + maggma: atomate2 stores results in MongoDB documents via the maggma store abstraction. Each job produces a document with fields for structure, energy, forces, stresses, magnetic moments, band gap, correction, and full provenance. The emmet package defines the document schema used by Materials Project. Direct MongoDB queries allow post-processing without re-running calculations.

When to Use This Skill

  • Systematically computing formation energies, band gaps, elastic moduli, or other properties across a library of hundreds to hundreds of thousands of structures.
  • Building internally consistent DFT datasets for ML potential training, surrogate model construction, or property prediction.
  • Running multi-step DFT pipelines (relax → static → DOS → phonon) automatically with dependency tracking.
  • Coupling DFT calculations to an active learning loop where new candidate structures are generated iteratively.
  • Setting up a group-internal materials database with consistent provenance for reproducible research.
  • Benchmarking universal MLPs or surrogate models against systematically computed DFT references on a held-out structure library.

When Not to Use This Skill

  • Single-material deep study: If the goal is one accurate number on one material (band gap, surface energy, defect level), use the individual DFT workflow skills directly. HT-DFT infrastructure adds overhead that is not justified for fewer than ~20 structures.
  • Mixed-DFT-code campaigns without explicit compatibility checking: Mixing VASP results with QE results, or PBE with SCAN, without a validated correction scheme produces a dataset that appears large but is internally inconsistent.
  • When the compute budget is extremely limited: HT-DFT pipelines are designed to handle thousands of jobs; setting up the infrastructure is not cost-effective for fewer than ~50 calculations.
  • When provenance recording is optional: If results will be published, presented, or used to train a model, provenance cannot be optional. If the team is unwilling to record and store full job metadata, HT-DFT at scale will produce untraceable results.
  • When no error handling strategy is planned: Unhandled DFT failures at scale silently corrupt datasets (missing structures, wrong convergence). Custodian or equivalent is non-optional for production campaigns.

Core Concepts

Full steps for each part are in references/core-concepts/; open the one you need:

Key Workflows

Full steps for each part are in references/key-workflows/; open the one you need:

Common Failure Modes

Failure ModeSymptomsLikely CauseDiagnosticFix
Silent job failureFIZZLED Firework; no custodian.json; OUTCAR missing or emptyVASP crashed before custodian could detect the error; OOM kill; filesystem issueCheck SLURM log: sacct -j <jobid>; check std_err.txtAdd StdErrHandler; increase memory request; check scratch disk quota
Inconsistent INCAR across batchEnergies non-comparable; phase diagram artifacts; suspicious outliersDifferent template INCARs used; manual edits; different InputSet versionsDiff INCARs across job directories; check pymatgen version in environmentPin pymatgen version; generate all INCARs from a single InputSet class at submission time
Mixed POTCAR versionsSystematic energy offsets by 0.1–1 eV; inconsistent formation energiesDifferent POTCAR library paths or dates used across nodesCheck POTCAR title lines (grep TITEL */POTCAR); compare hashesFix POTCAR library path; validate with potcar_validation.py; rerun affected jobs
Non-comparable energies across functionalFormation energies wrong sign or magnitude; phase diagram dominated by functional errorPBE and SCAN or PBE+U results mixed without correctionCheck GGA tag in INCAR; confirm all jobs use same functionalUse MaterialsProject2020Compatibility correction scheme; never mix raw energies across functionals
Duplicate structures in datasetModel overfits; benchmark scores inflated; misleading screening rankingCIF import, database duplicates, or symmetry-equivalent supercells not deduplicatedRun StructureMatcher on entire dataset; check for formula duplicates with different IDsDeduplicate before training and before train/test split using build_structure_groups()
Bad error handler causing infinite loopJob re-queued repeatedly; FIZZLED after max_errors; same error each relaunchError handler applies a fix that doesn't resolve the underlying errorCheck custodian.json for repeated corrections of same typeIncrease max_errors_per_job; investigate root cause; add a more specific handler
Unconverged relaxations acceptedForces > threshold; CONTCAR geometry incorrect; downstream static wrongNSW limit hit; EDIFFG too loose for material; custodian did not catchCheck ionic_steps == NSW in convergence check; inspect max force in OUTCARIncrease NSW; tighten EDIFFG; rerun failed relaxations with extended settings
Missing provenanceResults cannot be reproduced; journal requires full methodologyNo INCAR/KPOINTS/POTCAR archived; UUID not stored; database query not cachedTry to reconstruct inputs from results — if impossible, provenance is lostArchive inputs alongside results; store custodian.json with every job; record query parameters
Treating screening-tier as publication qualityEnergies off by 0.1–0.5 eV per atom vs high-accuracy; conclusions do not replicateCoarse k-mesh, low ENCUT, or loose EDIFF used for full campaignCheck tier label in DB; compare ENCUT and k-mesh to MP standardAdd tier_quality metadata field; never remove the tier label from any result
Corrupted MongoDB after failed writeMissing documents; partial document fields; duplicate UUIDsJob wrote partial output before crash; two jobs wrote same UUIDQuery for documents with missing required fields; check UUID uniquenessUse upsert=True with UUID as key; run database integrity checks after each batch
Provenance chain broken by re-relaxationIt is unclear which relaxed structure produced a static or phonon resultRe-relaxation output stored under same job name; UUID not propagatedCheck prev_dir fields in job inputs; trace UUID chainAlways propagate parent UUID to child jobs; store parent_uuid in every job document

Best Practices

Workflow design:

  • Define every workflow step in code, not in manually edited files. A workflow whose inputs were edited by hand cannot be reproduced.
  • Use InputSet classes exclusively for INCAR, KPOINTS, and POTCAR generation. Never manually edit individual INCAR files in a production campaign.
  • Fix the pymatgen, atomate2, custodian, and jobflow versions at the start of a campaign and do not upgrade mid-campaign. Log the conda environment as part of the campaign's provenance record.
  • Assign a UUID to every job at submission time (not at completion); use it as the database primary key throughout.

Error handling:

  • Always wrap VASP with custodian in production. An uncaught VASP error silently leaves no result, which looks identical to a job that was never submitted.
  • Set max_errors=5 and max_errors_per_job=3; jobs that exceed this are FIZZLED and available for manual inspection. Do not set max_errors to an arbitrarily large number to force completion.
  • Include WalltimeHandler for all VASP jobs on HPC. Without it, wall-time kills leave no WAVECAR and the job must restart from scratch.
  • Review FIZZLED jobs before requeuing; some failure modes (wrong structure, bad initial geometry) will recur regardless of the fix applied.

Convergence and quality:

  • Label every result with its convergence tier (screening, standard, high_accuracy) in the database and in any derived dataset.
  • Never mix results from different tiers in the same phase diagram, ML training set, or comparative analysis without an explicit compatibility correction.
  • Run convergence_policy.py (Workflow 6) on every completed batch before downstream use. Flag jobs where ionic steps == NSW or max force > threshold.

Deduplication and compatibility:

  • Deduplicate candidate structures before submission (Workflow 5) to avoid wasted compute.
  • Deduplicate again after ingestion, using completed-structure geometries (post-relaxation), which may have symmetrized or changed space group.
  • For thermodynamic analysis, apply MaterialsProject2020Compatibility to all energies to make them compatible with the MP database. Do not mix corrected and uncorrected energies.

HPC and scaling:

  • Use SLURM array jobs with a capped concurrency (--array=1-N%M) to avoid overwhelming the scheduler or exceeding fair-share allocations.
  • Set scratch directories per job on fast local storage; do not write large binary files (WAVECAR, CHGCAR) to network filesystems during the run.
  • Archive completed results (OUTCAR, vasprun.xml, custodian.json, CONTCAR) to long-term storage immediately after parsing. Do not rely on scratch disk for permanence.

Integration with Other Skills

Upstream DFT workflow skills (each workflow type is a branch of the HT-DFT graph):

  • VASP Workflow — the relaxation, static, and band structure jobs at each graph node use the same INCAR conventions and POTCAR selection protocol as the single-material VASP workflow skill. MPRelaxSet and MPStaticSet are the standard input sets for both.
  • Quantum ESPRESSO Workflow — QE-based HT-DFT workflows use AiiDA (via aiida-quantumespresso) instead of atomate2; the same convergence-tier and provenance principles apply.
  • DFT Convergence — convergence tier definitions (ENCUT, k-mesh, EDIFF thresholds) are derived from the convergence skill. Phonon and defect branches use high-accuracy tier settings from that skill.
  • DFT Phonons — phonon workflow branches in HT-DFT use atomate2's PhononMaker (wrapping Phonopy + VASP). Force constant calculations follow the same supercell and INCAR discipline.
  • DFT Surfaces — surface energy and adsorption energy branches use slab geometry conventions and dipole correction settings from the surfaces skill. Adsorption screening across many adsorbate/site combinations is a natural HT-DFT use case.
  • DFT Defects — defect formation energy screening across host materials or dopant species uses doped's DefectsSet embedded in a jobflow Flow. Image-charge corrections are applied in post-processing via the same doped/pydefect infrastructure.
  • DFT Dataset Generation — HT-DFT is the primary engine for generating large DFT reference datasets. This skill provides the orchestration layer; the dataset generation skill covers the structure sampling, labeling format, and train/test split.

Data and analysis infrastructure:

  • Pymatgen Analysis — StructureMatcher, InputSet classes, ComputedEntry, PhaseDiagram, and ChemicalPotentialDiagram are used throughout HT-DFT for structure matching, input generation, and thermodynamic analysis. atomate2 is built on pymatgen.
  • ASE Framework — structure import/export, extxyz dataset generation, and MLP evaluation on HT-DFT structures all use ASE as the interoperability layer.
  • Materials Databases Access — HT-DFT campaigns often start with structures retrieved from the Materials Project, AFLOW, or OQMD. Compatible energy references and correction schemes from this skill ensure that local HT-DFT results can be compared against database values.
  • Materials Featurization — after HT-DFT, computed properties are featurized for surrogate model training. This skill provides the bridge from DFT results to ML-ready feature vectors.
  • Reproducibility Workflow — HT-DFT campaigns are the highest-stakes context for reproducibility. Campaign inputs (InputSet parameters, POTCAR library, queue configuration, environment), results (MongoDB snapshot), and analysis scripts must all be version-controlled and archived together.

ML and screening downstream:

  • Active Learning MLP — Workflow 7 implements the interface between HT-DFT results and an active learning loop. Completed DFT results update the surrogate, which selects the next DFT batch.
  • Gaussian Process Materials — GP surrogates are a common choice for property screening with uncertainty quantification; the HT-DFT dataset is the training set and the surrogate ranks uncomputed candidates. [FORWARD REFERENCE — skill may be in progress]
  • Bayesian Optimization — Bayesian optimization over material space uses HT-DFT results as the objective function evaluator; the surrogate (GP or neural network) and acquisition function determine the next batch. [FORWARD REFERENCE — skill not yet created]

Key References

  1. Jain, A., Ong, S. P., Hautier, G., Chen, W., Richards, W. D., Dacek, S., Cholia, S., Gunter, D., Skinner, D., Ceder, G. & Persson, K. A. (2013). "Commentary: The Materials Project: A materials genome approach to accelerating materials innovation." APL Materials 1, 011002. The Materials Project HT-DFT database and infrastructure reference.
  2. Mathew, K., Montoya, J. H., Faghaninia, A., Dwaraknath, S., Aykol, M., Tang, H., Chu, I., Smidt, T., Bocklund, B., Horton, M., Dagdelen, J., Wood, B., Liu, Z.-K., Neaton, J., Ong, S. P., Persson, K. & Jain, A. (2017). "Atomate: A high-level interface to generate, execute, and analyze computational materials science workflows." Computational Materials Science 139, 140–152. atomate (v1) workflow infrastructure reference.
  3. Ganose, A. M., Sai Gautam, G., Sherrill, C. D. & Jain, A. (2024). "Atomate2: A high-throughput materials science workflow management framework." Journal of Open Source Software (in press/preprint). atomate2 reference; cite when using atomate2.
  4. Ong, S. P., Richards, W. D., Jain, A., Hautier, G., Kocher, M., Cholia, S., Gunter, D., Chevrier, V. L., Persson, K. A. & Ceder, G. (2013). "Python Materials Genomics (pymatgen): A robust, open-source python library for materials analysis." Computational Materials Science 68, 314–319. pymatgen reference.
  5. Rosen, A. S., Gallant, M., George, J., Riebesell, J., Sasaki, H., Shen, J.-X., Sherrill, C. D. & Ganose, A. M. (2024). "jobflow: Computational workflows made simple." Journal of Open Source Software 9, 5995. jobflow reference.
  6. Boes, J. R., Groenenboom, M. C., Keith, J. A. & Kitchin, J. R. (2016). "Neural network and ReaxFF comparison for Au properties." International Journal of Quantum Chemistry 116, 979–987. Example of custodian-style error handling in automated DFT.
  7. Curtarolo, S., Setyawan, W., Hart, G. L. W., Jahnatek, M., Chepulskii, R. V., Taylor, R. H., Wang, S., Xue, J., Yang, K., Levy, O., Mehl, M. J., Stokes, H. T., Demchenko, D. O. & Morgan, D. (2012). "AFLOW: An automatic framework for high-throughput materials discovery." Computational Materials Science 58, 218–226. AFLOW HT-DFT framework; alternative to atomate2.
  8. Pizzi, G., Cepellotti, A., Sabatini, R., Marzari, N. & Kozinsky, B. (2016). "AiiDA: Automated interactive infrastructure and database for computational science." Computational Materials Science 111, 218–230. AiiDA provenance-first HT-DFT infrastructure; preferred for cross-institutional reproducibility.

[EXPERT REVIEW NEEDED: Verify that atomate2 version 0.0.14 is still current; the package is under active development and API changes frequently. Confirm whether MaterialsProject2020Compatibility should be applied to SCAN results or only to GGA/GGA+U results. Verify the custodian handler list covers all common VASP 6.x failure modes, particularly new error strings introduced in VASP 6.3+.]

示例提示词

我想用atomate2搭建大规模自动化DFT筛选工作流并追踪计算溯源。

所属插件

计算材料学:第一性原理与原子模拟面向计算材料研究:DFT 与第一性原理(VASP、Quantum ESPRESSO、CP2K、AiiDA)、缺陷/表面/声子/弹性常数、机器学习势函数训练与验证、分子动力学与增强采样、晶体性质图神经网络与生成模型。

安装该插件会一并获得同领域的 35 个 Skill / MCP。

同插件的其他科研 Skill

常见问题

高通量DFT自动化筛选能做什么?

Covers automated large-scale DFT screening: workflow graphs with atomate2/jobflow/AiiDA, input-set standardization, custodian error handling. Use for high-throughput DFT screening campaigns。

如何安装高通量DFT自动化筛选?

在科搭 Codar 左侧点击「插件」,在「组件」页签搜索「高通量DFT自动化筛选」并安装;在 Claude Code 或 Codex 中可以发送本页的安装提示词,或下载 Skill 包后解压到 ~/.claude/skills 或 ~/.codex/skills。

高通量DFT自动化筛选需要哪些权限?

该科研 Skill没有声明额外权限;执行具体操作时仍由 Agent 按自身规则请求确认。

高通量DFT自动化筛选经过官方审核吗?

是。SpeedSkills 收录的每个版本都会记录内容摘要与权限声明,当前版本已通过安全审核。