科研 Skill
AiiDA计算工作流管理
aiida-workflow
Covers AiiDA, a provenance-first Python workflow engine for computational materials science: calculation input generation, remote submission. Use when orchestrating reproducible DFT/MD campaigns.
SKILL.md
AiiDA Workflow
Description
AiiDA (Automated Interactive Infrastructure and Database for Computational Science) is a Python-based, provenance-first workflow engine for computational materials science. It manages the complete lifecycle of calculations: input generation, remote HPC job submission via SSH, output retrieval and parsing, automatic error handling and restart, and immutable provenance recording in a directed acyclic graph (DAG) of data and process nodes. Invoke this skill when designing automated, reproducible DFT or AIMD workflows for long-term campaigns, when cross-institutional provenance is required, when workflows must survive HPC failures and restart automatically, or when producing data to be archived to external repositories (NOMAD, Materials Cloud) with machine-readable, verifiable calculation histories.
Domain Context
AiiDA's central design decision is that all data and all processes are nodes in a provenance graph. Every calculation input, every output, and every transformation is stored in a relational database (PostgreSQL or SQLite) as an immutab…
Moved to references/domain-context.md to keep this file short; read it when this part is needed.
When to Use This Skill
- Long-term, cross-institutional, or publication-grade HT-DFT campaigns where every result must be auditable with full input–output linkage and verifiable provenance.
- Workflows that must survive HPC failures, wall-time kills, or network interruptions: AiiDA's WorkChain checkpointing and daemon restart logic handle these automatically.
- Multi-code campaigns (QE + CP2K + VASP) requiring a unified provenance graph across codes.
- When results must be shared via NOMAD, Materials Cloud, or AiiDA Archive — the
.aiidaarchive format is self-contained and importable on any AiiDA installation. - Phonon calculations using
aiida-quantumespresso'sPhononWorkChain(dozens of DFPT or displaced-supercell CalcJobs benefit from automatic management and provenance). - When a code-agnostic interface is needed:
aiida-common-workflowsprovidesCommonRelaxWorkChainfor structure relaxation with QE, VASP, CP2K, or GPAW under a single unified API.
When Not to Use This Skill
- Short campaigns (fewer than ~50 calculations) on a single system: Profile, computer, code, and daemon setup overhead is not recovered. Use ASE calculators with a SLURM array script instead.
- When turnaround time is the primary constraint: AiiDA's daemon polling interval and node-creation overhead add latency compared to direct SLURM submission. For urgent exploratory calculations, submit directly.
- When the team cannot maintain a daemon and PostgreSQL server: AiiDA requires persistent processes. HPC environments that disallow long-running processes on login nodes need a gateway machine or container setup. SQLite mode exists but does not support the daemon and serializes all work.
- When atomate2 + MongoDB is already the group standard: Migrating an atomate2 campaign midway to AiiDA is expensive. Use atomate2 for code that already uses it; use AiiDA for new campaigns where provenance depth justifies the setup.
- Workflows requiring sub-minute task granularity: The daemon polling interval (default every few seconds) and per-node database writes add overhead disproportionate to tasks completing in under a minute.
Core Concepts
- Profile: A named AiiDA installation instance binding a PostgreSQL database, Disk-Objectstore repository, and daemon. Switch with
verdi profile setdefault <name>or theAIIDA_PROFILEenvironment variable. AiiDA 2.x supportscore.psql_dos(PostgreSQL, production) andcore.sqlite_dos(SQLite, local testing). - Daemon: Worker processes that execute the submission–poll–retrieve loop. Must be running for submitted calculations to advance.
verdi daemon start 2launches 2 workers;verdi daemon statusshows their state. WorkChain checkpoints survive daemon restarts; they do not survive PostgreSQL data loss. - verdi CLI: The AiiDA command-line interface. Key commands:
verdi setup/verdi quicksetup(profile creation),verdi computer setup+verdi computer configure(HPC resource registration),verdi code create core.code.installed(code registration),verdi daemon start|stop|status|restart,verdi process list(active and recent processes),verdi node show <UUID|PK>(node inspection),verdi calcjob gotocomputer <PK>(SSH to job directory),verdi archive export(provenance export). - Computer: An AiiDA object representing a remote HPC resource. Stores hostname, SSH username, transport plugin, scheduler type, work directory, and default MPI command. Configure once per HPC system; test with
verdi computer test <name>. Transport plugins:core.local(for local execution),core.ssh(for remote HPC via SSH). - Code: An AiiDA object representing an executable installed on a Computer. Stores the absolute path to the binary, the AiiDA plugin entry point (e.g.,
quantumespresso.pw), and prepend/append shell text for module loading. A separate Code node is required for each executable–computer combination. - StructureData: AiiDA's crystal structure data node. Stores unit cell vectors and atomic sites. Converts via
structure.get_ase()(returns ASEAtoms) andstructure.get_pymatgen()(returns pymatgenStructure). Constructed fromStructureData(ase=atoms)orStructureData(pymatgen=struct). Immutable once stored — modifying a structure requires creating a new node. - Dict: AiiDA's dictionary node for parameters. Used for all DFT input parameters (QE namelists, CP2K input sections, VASP INCAR-equivalent dicts). Must be JSON-serializable. Nested dicts are allowed.
- KpointsData: Stores a k-mesh (Monkhorst-Pack), explicit k-point list with weights, or high-symmetry path.
kpoints.set_kpoints_mesh([6, 6, 6])sets a 6×6×6 grid centered at Γ.kpoints.set_kpoints_mesh_from_density(0.1)generates a mesh automatically from a reciprocal-space density in Å⁻¹. - ProcessBuilder: Interface for setting up a CalcJob or WorkChain.
Process.get_builder()returns it; inputs are set as attributes with type validation.submit(builder)submits asynchronously to the daemon;run(builder)executes synchronously (blocking, no daemon needed, for testing). - WorkChain outline: The step sequence, defined in
cls.define(spec)viaspec.outline(cls.step1, if_(cls.condition)(cls.step2), cls.step3). Each step is a checkpoint — the WorkChain persists its context before each step, enabling daemon-restart recovery.self.ctxis the persistent context dictionary. - ExitCode: An integer exit condition attached to a WorkChain class via
spec.exit_code(code, 'LABEL', message='...'). Returned from a step asreturn self.exit_codes.LABELto terminate the WorkChain with that status. Calling code querieswc.exit_statusandwc.exit_message. - QueryBuilder: AiiDA's graph query engine. Supports filtering by node type, attributes, extras, and link labels; joining across node types; and projecting specific fields. More powerful than MongoDB for graph traversal; comparable to SQL for scalar fields.
- AiiDA Archive: A portable
.aiidazip file that contains all nodes and links reachable from a set of root nodes (traversing the provenance graph automatically). Created withverdi archive export; imported withverdi archive import. Used for sharing data with collaborators and uploading to NOMAD or Materials Cloud.
Key Workflows
Full steps for each part are in references/key-workflows/; open the one you need:
- Workflow 1: Profile setup, computer registration, code configuration, and daemon startup —
references/key-workflows/01-workflow-1-profile-setup-computer-registration-c.md: Setting up AiiDA for a new project requires creating a profile, registering the remote HPC computer, configuring SSH transport, installing pseudopotentials, registering the DFT code, and starting the daemon. This is a one-time setup per pr… - Workflow 2: Single QE SCF calculation as a CalcJob (ProcessBuilder) —
references/key-workflows/02-workflow-2-single-qe-scf-calculation-as-a-calcjo.md: APwCalculationCalcJob wraps a singlepw.xexecution. All inputs are passed as AiiDA data nodes via aProcessBuilder. This workflow shows the minimal complete setup; in production always wrap in a WorkChain (Workflow 3). - Workflow 3: PwBaseWorkChain — automatic error handling and restart —
references/key-workflows/03-workflow-3-pwbaseworkchain-automatic-error-handl.md:PwBaseWorkChainwrapsPwCalculationwith automatic error detection and recovery. It handles common QE failures (wall-time expiry, electronic convergence failures, CRASH, bad parallelisation) by adjusting settings and restarting. Always… - Workflow 4: k-point convergence study using AiiDA and QueryBuilder —
references/key-workflows/04-workflow-4-k-point-convergence-study-using-aiida.md: This workflow submits a series of SCF calculations at increasing k-mesh densities, tags them with extras for easy retrieval, and queries the provenance graph to build a convergence table after completion. - Workflow 5: QueryBuilder — extracting results and exporting provenance —
references/key-workflows/05-workflow-5-querybuilder-extracting-results-and-e.md: QueryBuilder is AiiDA's graph traversal and query engine. It replaces MongoDB queries and direct database introspection for extracting completed-calculation results, quality-checking a campaign, and preparing data for export. - Workflow 6: High-throughput relaxation campaign —
references/key-workflows/06-workflow-6-high-throughput-relaxation-campaign.md: A production HT-DFT campaign submits hundreds of structures asPwRelaxWorkChainjobs, organises them in an AiiDA Group for bulk monitoring and export, and restarts idempotently (skipping already-submitted structures on re-run).
Common Failure Modes
Moved to references/common-failure-modes.md to keep this file short; read it when this part is needed.
Best Practices
- Store all inputs as AiiDA nodes before submission.
submit(builder)stores unstored nodes automatically, but calling.store()explicitly gives you the PK/UUID immediately for cross-referencing with external systems. - Use Groups to organise every campaign. Create one Group per campaign (
Group.objects.get_or_create(label=campaign_label)) and add every submitted process and its input structure. This enablesverdi process list --group, bulk QueryBuilder filtering by group, and clean bulk export. Never rely solely on label-prefix matching. - Always use WorkChains, not raw CalcJobs, in production.
PwBaseWorkChain,Cp2kBaseWorkChain, and their equivalents implement error handling, restart logic, and exit codes that raw CalcJobs lack. A raw CalcJob that hits a wall-time limit leaves aFAILEDnode with no attempt at recovery. - Pin
aiida-coreand plugin versions per project. Database schemas and plugin APIs change across minor versions. Record the pinned versions inrequirements.txt, commit it alongside the campaign setup scripts, and runverdi storage migrateon a database copy before any upgrade. - Set
clean_workdir=Truein production campaigns. AiiDA stores the files it needs in its repository automatically during retrieval. The remote HPC scratch directory can and should be deleted after that. Without this, scratch quotas fill up silently over long campaigns. - Tag every submitted process with
extras.wc.set_extra("formula", formula)andwc.set_extra("campaign", campaign_label)enable QueryBuilder filtering andverdi process listgrouping. Label strings alone are fragile; extras are queryable fields. - Use
aiida-pseudofor all pseudopotential management. Pseudopotentials installed viaaiida-pseudo install ssspare stored asUpfDatanodes in the AiiDA repository with checksums, are included in exported archives, and can be imported on other machines with matching UUIDs. Manually placed UPF files referenced by local path break portability and reproducibility. - Export provenance archives early and regularly.
verdi archive export -O campaign.aiida -G <group>creates a portable, importable snapshot. Export after each milestone (cutoff convergence, relaxations complete, phonons complete). A database corruption between exports loses only the most recent work. Archives are immutable; they cannot be accidentally edited. - Validate computer configuration after any network or HPC change.
verdi computer test <computer>verifies SSH connectivity, shell availability, scheduler command parsing, and working directory access. Run it whenever HPC hostnames, SSH keys, module paths, or scheduler configurations change. A failing computer test explains mostSUBMIT_FAILEDerrors.
Integration with Other Skills
- high-throughput-dft: AiiDA is an alternative to atomate2/jobflow for HT-DFT orchestration, with stronger provenance guarantees and better cross-institutional portability. The convergence tier taxonomy (screening vs. standard vs. high-accuracy), quality flag methodology, and deduplication strategy from that skill apply equally to AiiDA-managed campaigns. AiiDA and atomate2 can coexist in the same research group: use atomate2 for rapid internal campaigns, AiiDA for publication-grade or cross-institutional work.
- quantum-espresso:
aiida-quantumespressowraps QE'spw.x,ph.x,nscf.x, andpp.xas AiiDA CalcJobs. Theecutwfc,ecutrho,degauss,ibrav=0, and pseudopotential selection rules from the QE skill are the domain knowledge that must be encoded in theDictparameters passed toPwCalculationandPwBaseWorkChain. The SSSP pseudopotential family installed viaaiida-pseudocorresponds directly to the SSSP library described in that skill. - cp2k-workflow:
aiida-cp2kprovidesCp2kCalculationandCp2kBaseWorkChainfor running CP2K DFT and AIMD workflows under AiiDA provenance. CP2K input files are constructed as nested Python dicts (converted to&SECTIONformat by the plugin). CP2K is the preferred AiiDA backend for large-scale AIMD and condensed-phase simulations. The CUTOFF, REL_CUTOFF, and OT/diagonalization settings from the CP2K skill must be encoded as explicit Dict nodes, not embedded as plugin defaults. - vasp-workflow:
aiida-vaspprovides VASP workflow support under AiiDA. VASP's proprietary license means POTCAR files must be installed manually on each Computer and are not redistributable viaaiida-pseudo. pymatgenVaspInputSetclasses can generate INCAR and KPOINTS, which are then wrapped as Dict nodes before submission. [EXPERT REVIEW NEEDED: verify aiida-vasp compatibility with AiiDA 2.5 and VASP 6.4] - dft-convergence: The cutoff scan, k-mesh convergence, and smearing checks from the convergence skill are the domain foundation for the AiiDA parameter sweeps in Workflow 4. AiiDA preserves the full convergence series as a set of linked WorkChainNodes sharing a common StructureData input node, making the chosen production parameters auditable in the provenance graph without requiring a separate convergence log.
- dft-dataset-generation: AiiDA provides a provenance-tracked alternative to ad hoc SLURM arrays for DFT training data generation. Workflow 5 above extracts energy, forces, and stresses from
PwCalculationoutput nodes and writes them to extxyz with the CalcJob UUID as a per-frame identifier, satisfying the dataset-level provenance requirements described in that skill. - materials-databases-access: Starting structures for AiiDA campaigns are typically retrieved from the Materials Project via
mp-apior from AFLOW, COD, or ICSD. The retrieved pymatgenStructureobjects are immediately converted toStructureDatanodes and stored before submission, establishing a direct provenance link from the external database record to the AiiDA calculation. The ICSD and COD identifiers can be stored as node extras for traceability. - pymatgen-analysis:
StructureData.get_pymatgen()andStructureData(pymatgen=...)are the primary interfaces between AiiDA's provenance graph and pymatgen's analysis tools. Phase diagram construction viaPhaseDiagram, structure symmetrization viaSpacegroupAnalyzer, and convex hull stability analysis all happen on pymatgen objects extracted from AiiDA via QueryBuilder. - ase-framework:
StructureData.get_ase()andStructureData(ase=atoms)are the low-overhead bridge for structure manipulation and extxyz export. The ASE interface is preferred over pymatgen for bulk trajectory export from AiiDA (Workflow 5) due to lower conversion overhead for large frame counts. - reproducibility-workflow: AiiDA's provenance graph is the strongest available reproducibility tool for DFT campaigns — the graph itself is the provenance record. The complementary layer from that skill (environment snapshots, pseudopotential checksums, DVC for exported archives) applies to the AiiDA setup: pin and record
aiida-coreand plugin versions inrequirements.txt; checksum AiiDA archive files with SHA-256 before depositing to NOMAD; version-controlverdi computerandverdi codeconfiguration YAML files alongside campaign setup scripts. - scientific-visualization:
BandsDatanodes fromPwBandsWorkChainare visualized with pymatgen'sBSPlotteror viaverdi data bands export --format gnuplot. Phonon dispersion fromPhononWorkChainis exported as aBandsDatanode and plotted with phonopy. Relaxation trajectories stored asTrajectoryDatanodes are exported to extxyz and visualized with OVITO. - mlp-validation: DFT reference data in AiiDA (energies, forces, stresses from
PwCalculationorCp2kCalculationoutput nodes) can be exported as extxyz test sets for MLP validation (Workflow 5). The CalcJob UUID provides a unique, traceable identifier for each frame, enabling exact traceability from an MLP validation error back to the specific DFT calculation and its full provenance chain.
Key References
- Pizzi, G., Cepellotti, A., Sabatini, R., Marzari, N. & Kozinsky, B. (2016). AiiDA: Automated interactive infrastructure and database for computational science. Computational Materials Science, 111, 218–230. https://doi.org/10.1016/j.commatsci.2015.09.013
- Huber, S. P., Zoupanos, S., Uhrin, M., Talirz, L., Kahle, L., Häuselmann, R., Gresch, D., Müller, T., Yakutovich, A. V., Andersen, C. W., Ramirez, F. F., Adorf, C. S., Gargiulo, F., Kumbhar, S., Passaro, E., Johnston, C., Merkys, A., Cepellotti, A., Mounet, N., Marzari, N., Kozinsky, B. & Pizzi, G. (2020). AiiDA 1.0, a scalable computational infrastructure for automated reproducible workflows and data provenance in computational science. Scientific Data, 7, 300. https://doi.org/10.1038/s41597-020-00638-4
- Uhrin, M., Huber, S. P., Yu, J., Marzari, N. & Pizzi, G. (2021). Workflows in AiiDA: Engineering a high-throughput, event-based engine for robust and modular computational workflows. Computational Materials Science, 187, 110086. https://doi.org/10.1016/j.commatsci.2020.110086
- Huber, S. P., Bosoni, E., Bercx, M., Bröder, J., Degomme, A., Dikan, V., Eimre, K., Flament, M., García, A., Grisafi, A., Guda, S., Hourahine, B., Huran, A. W., Illas, F., Keller, M., Laskowski, R., Lin, L., Mercado, R., Mostofi, A. A., Petretto, G., Ratcliff, L. E., Rignanese, G.-M., Schütt, O., Timrov, I., Torrent, M., Van Hove, M., Wirtz, L., Zhang, C., Normand, J., Marzari, N., Goedecker, S., Needs, R. J., Marzari, N. & Pizzi, G. (2021). Common workflows for computing material properties using different quantum engines. npj Computational Materials, 7, 291. https://doi.org/10.1038/s41524-021-00594-6
- Bosoni, E., Beal, L., Bercx, M., Blöchl, P., Bühlmann, M., Cappi, F., Castelli, I., Cepellotti, A., Cerqueira, T., Corsetti, F., & Pizzi, G. (2024). How to verify the precision of density-functional-theory implementations via reproducible and universal workflows. Nature Reviews Physics, 6, 45–58. https://doi.org/10.1038/s42254-023-00655-3
- Talirz, L., Kumbhar, S., Passaro, E., Yakutovich, A. V., Granata, V., Gargiulo, F., Borelli, M., Uhrin, M., Huber, S. P., Zoupanos, S., Adorf, C. S., Andersen, C. W., Schütt, O., Pignedoli, C. A., Passerone, D., VandeVondele, J., Schulthess, T. C., Smit, B., Pizzi, G. & Marzari, N. (2020). Materials Cloud, a platform for open computational science. Scientific Data, 7, 299. https://doi.org/10.1038/s41597-020-00637-5
示例提示词
帮我用AiiDA搭建一套可追溯的DFT计算工作流并提交到HPC集群。
所属插件
安装该插件会一并获得同领域的 35 个 Skill / MCP。
同插件的其他科研 Skill
- 主动学习构建机器学习势训练集Runs active-learning workflows for ML interatomic potentials: uncertainty-guided MD exploration, DFT oracle labeling, query strategy. Use when expanding an MLP training set.
- ASE原子模拟环境Covers the Atomic Simulation Environment (ASE) as an atomistic workflow hub: Atoms objects, calculators, optimizers, trajectories. Use when scripting or converting structures across DFT/MD codes.
- 粗粒化分子动力学模拟Covers coarse-grained molecular dynamics: mapping atomistic structures to CG beads, parameterizing Martini/IBI/force-matching force fields. Use when atomistic timescales are too short.
- CP2K第一性原理分子动力学Covers end-to-end DFT, AIMD, and atomistic workflows with CP2K using the Gaussian-and-plane-waves method: input construction, convergence testing. Use for CP2K-based simulations.
- 扩散模型生成晶体结构Covers de novo crystal structure generation with score-based diffusion and flow-matching models (MatterGen, DiffCSP, CDVAE). Use when generating candidate structures, not analyzing existing ones.
- 晶体图神经网络性质预测Covers graph neural networks for crystal and periodic-material property prediction: crystal graph construction, periodic neighbor finding. Use when building GNN property predictors for crystals.
- 平面波DFT收敛性测试Covers systematic convergence testing for plane-wave DFT: cutoff energy, k-point mesh density, smearing. Use before trusting DFT energies, forces, or derived properties as converged.
- 机器学习势DFT数据集生成Covers generating DFT-labeled datasets for training ML interatomic potentials: configuration-space coverage. Use when building an MLP training dataset from scratch.
- 点缺陷第一性原理计算Covers first-principles point-defect modeling: vacancy/interstitial/antisite/substitutional construction, charge-state calculations. Use for defect thermodynamics studies.
- 第一性原理声子谱计算Covers lattice dynamics from first principles: phonon dispersion, phonon DOS. Use when computing vibrational or thermodynamic properties from DFT.
- 表面模型DFT计算Covers surface slab model construction and DFT calculation: Miller-index generation, termination selection, slab/vacuum convergence. Use for surface science and catalysis DFT studies.
- 弹性常数计算工作流Computes single-crystal and polycrystalline elastic constants from DFT or MD stress-strain data and checks mechanical stability. Use when deriving stiffness tensors or bulk/shear modulus.
- 增强采样自由能计算Covers enhanced sampling methods for atomistic MD (metadynamics, umbrella sampling. Use for rare-event or slow-transition sampling.
- 状态方程拟合Covers equation-of-state fitting from DFT/ML-potential energy-volume data: Birch-Murnaghan, Vinet, and Murnaghan forms. Use when fitting an EOS to compression data.
- 机器学习势微调迁移训练Covers fine-tuning pretrained universal ML interatomic potentials on system-specific DFT data. Use when adapting a universal MLP to a new system.
- 分子动力学自由能计算Covers quantitative free-energy calculations from MD: alchemical FEP/TI/BAR/MBAR, umbrella-sampling PMF. Use when computing thermodynamic free energies from simulation.
- GROMACS分子动力学工作流Covers end-to-end MD simulations with GROMACS: topology/.mdp construction, energy minimization, NVT/NPT equilibration. Use for GROMACS-based biomolecular or materials MD.
- 高通量DFT自动化筛选Covers automated large-scale DFT screening: workflow graphs with atomate2/jobflow/AiiDA, input-set standardization, custodian error handling. Use for high-throughput DFT screening campaigns.
- LAMMPS分子动力学工作流Covers end-to-end MD simulations with LAMMPS: input script anatomy, force-field deployment, ensemble control, equilibration. Use for LAMMPS-based classical molecular dynamics.
- MACE机器学习势训练Covers training MACE machine-learned interatomic potentials: extxyz dataset preparation, isolated-atom energies, train/val/test splitting. Use for MACE-specific MLP training workflows.
- MatGL图神经网络框架Covers building, applying, validating, and fine-tuning MatGL graph-neural-network models for property prediction and interatomic potentials. Use for pretrained GNN workflows via MatGL.
- 分子动力学轨迹后处理分析Covers quantitative post-processing of atomistic MD trajectories: loading LAMMPS/GROMACS/ASE trajectories and computing RDF, coordination numbers. Use for structural MD trajectory analysis.
- 分子动力学平衡过程设计Covers bringing an atomistic simulation from its initial configuration to a representative equilibrium state before production statistics: ensemble. Use before collecting production MD data.
- 分子动力学力场选型验证Covers selection, validation, and deployment of classical interatomic force fields for MD: physical basis and transferability limits of EAM, MEAM, Tersoff. Use when choosing a force field.
常见问题
AiiDA计算工作流管理能做什么?
Covers AiiDA, a provenance-first Python workflow engine for computational materials science: calculation input generation, remote submission. Use when orchestrating reproducible DFT/MD campaigns。
如何安装AiiDA计算工作流管理?
在科搭 Codar 左侧点击「插件」,在「组件」页签搜索「AiiDA计算工作流管理」并安装;在 Claude Code 或 Codex 中可以发送本页的安装提示词,或下载 Skill 包后解压到 ~/.claude/skills 或 ~/.codex/skills。
AiiDA计算工作流管理需要哪些权限?
该科研 Skill没有声明额外权限;执行具体操作时仍由 Agent 按自身规则请求确认。
AiiDA计算工作流管理经过官方审核吗?
是。SpeedSkills 收录的每个版本都会记录内容摘要与权限声明,当前版本已通过安全审核。