科研 Skill

AiiDA计算工作流管理

aiida-workflow

Covers AiiDA, a provenance-first Python workflow engine for computational materials science: calculation input generation, remote submission. Use when orchestrating reproducible DFT/MD campaigns.

发布者 Codar安全审核通过v0.1.0更新于 2026-11-18

SKILL.md

AiiDA Workflow

Description

AiiDA (Automated Interactive Infrastructure and Database for Computational Science) is a Python-based, provenance-first workflow engine for computational materials science. It manages the complete lifecycle of calculations: input generation, remote HPC job submission via SSH, output retrieval and parsing, automatic error handling and restart, and immutable provenance recording in a directed acyclic graph (DAG) of data and process nodes. Invoke this skill when designing automated, reproducible DFT or AIMD workflows for long-term campaigns, when cross-institutional provenance is required, when workflows must survive HPC failures and restart automatically, or when producing data to be archived to external repositories (NOMAD, Materials Cloud) with machine-readable, verifiable calculation histories.

Domain Context

AiiDA's central design decision is that all data and all processes are nodes in a provenance graph. Every calculation input, every output, and every transformation is stored in a relational database (PostgreSQL or SQLite) as an immutab…

Moved to references/domain-context.md to keep this file short; read it when this part is needed.

When to Use This Skill

  • Long-term, cross-institutional, or publication-grade HT-DFT campaigns where every result must be auditable with full input–output linkage and verifiable provenance.
  • Workflows that must survive HPC failures, wall-time kills, or network interruptions: AiiDA's WorkChain checkpointing and daemon restart logic handle these automatically.
  • Multi-code campaigns (QE + CP2K + VASP) requiring a unified provenance graph across codes.
  • When results must be shared via NOMAD, Materials Cloud, or AiiDA Archive — the .aiida archive format is self-contained and importable on any AiiDA installation.
  • Phonon calculations using aiida-quantumespresso's PhononWorkChain (dozens of DFPT or displaced-supercell CalcJobs benefit from automatic management and provenance).
  • When a code-agnostic interface is needed: aiida-common-workflows provides CommonRelaxWorkChain for structure relaxation with QE, VASP, CP2K, or GPAW under a single unified API.

When Not to Use This Skill

  • Short campaigns (fewer than ~50 calculations) on a single system: Profile, computer, code, and daemon setup overhead is not recovered. Use ASE calculators with a SLURM array script instead.
  • When turnaround time is the primary constraint: AiiDA's daemon polling interval and node-creation overhead add latency compared to direct SLURM submission. For urgent exploratory calculations, submit directly.
  • When the team cannot maintain a daemon and PostgreSQL server: AiiDA requires persistent processes. HPC environments that disallow long-running processes on login nodes need a gateway machine or container setup. SQLite mode exists but does not support the daemon and serializes all work.
  • When atomate2 + MongoDB is already the group standard: Migrating an atomate2 campaign midway to AiiDA is expensive. Use atomate2 for code that already uses it; use AiiDA for new campaigns where provenance depth justifies the setup.
  • Workflows requiring sub-minute task granularity: The daemon polling interval (default every few seconds) and per-node database writes add overhead disproportionate to tasks completing in under a minute.

Core Concepts

  • Profile: A named AiiDA installation instance binding a PostgreSQL database, Disk-Objectstore repository, and daemon. Switch with verdi profile setdefault <name> or the AIIDA_PROFILE environment variable. AiiDA 2.x supports core.psql_dos (PostgreSQL, production) and core.sqlite_dos (SQLite, local testing).
  • Daemon: Worker processes that execute the submission–poll–retrieve loop. Must be running for submitted calculations to advance. verdi daemon start 2 launches 2 workers; verdi daemon status shows their state. WorkChain checkpoints survive daemon restarts; they do not survive PostgreSQL data loss.
  • verdi CLI: The AiiDA command-line interface. Key commands: verdi setup/verdi quicksetup (profile creation), verdi computer setup + verdi computer configure (HPC resource registration), verdi code create core.code.installed (code registration), verdi daemon start|stop|status|restart, verdi process list (active and recent processes), verdi node show <UUID|PK> (node inspection), verdi calcjob gotocomputer <PK> (SSH to job directory), verdi archive export (provenance export).
  • Computer: An AiiDA object representing a remote HPC resource. Stores hostname, SSH username, transport plugin, scheduler type, work directory, and default MPI command. Configure once per HPC system; test with verdi computer test <name>. Transport plugins: core.local (for local execution), core.ssh (for remote HPC via SSH).
  • Code: An AiiDA object representing an executable installed on a Computer. Stores the absolute path to the binary, the AiiDA plugin entry point (e.g., quantumespresso.pw), and prepend/append shell text for module loading. A separate Code node is required for each executable–computer combination.
  • StructureData: AiiDA's crystal structure data node. Stores unit cell vectors and atomic sites. Converts via structure.get_ase() (returns ASE Atoms) and structure.get_pymatgen() (returns pymatgen Structure). Constructed from StructureData(ase=atoms) or StructureData(pymatgen=struct). Immutable once stored — modifying a structure requires creating a new node.
  • Dict: AiiDA's dictionary node for parameters. Used for all DFT input parameters (QE namelists, CP2K input sections, VASP INCAR-equivalent dicts). Must be JSON-serializable. Nested dicts are allowed.
  • KpointsData: Stores a k-mesh (Monkhorst-Pack), explicit k-point list with weights, or high-symmetry path. kpoints.set_kpoints_mesh([6, 6, 6]) sets a 6×6×6 grid centered at Γ. kpoints.set_kpoints_mesh_from_density(0.1) generates a mesh automatically from a reciprocal-space density in Å⁻¹.
  • ProcessBuilder: Interface for setting up a CalcJob or WorkChain. Process.get_builder() returns it; inputs are set as attributes with type validation. submit(builder) submits asynchronously to the daemon; run(builder) executes synchronously (blocking, no daemon needed, for testing).
  • WorkChain outline: The step sequence, defined in cls.define(spec) via spec.outline(cls.step1, if_(cls.condition)(cls.step2), cls.step3). Each step is a checkpoint — the WorkChain persists its context before each step, enabling daemon-restart recovery. self.ctx is the persistent context dictionary.
  • ExitCode: An integer exit condition attached to a WorkChain class via spec.exit_code(code, 'LABEL', message='...'). Returned from a step as return self.exit_codes.LABEL to terminate the WorkChain with that status. Calling code queries wc.exit_status and wc.exit_message.
  • QueryBuilder: AiiDA's graph query engine. Supports filtering by node type, attributes, extras, and link labels; joining across node types; and projecting specific fields. More powerful than MongoDB for graph traversal; comparable to SQL for scalar fields.
  • AiiDA Archive: A portable .aiida zip file that contains all nodes and links reachable from a set of root nodes (traversing the provenance graph automatically). Created with verdi archive export; imported with verdi archive import. Used for sharing data with collaborators and uploading to NOMAD or Materials Cloud.

Key Workflows

Full steps for each part are in references/key-workflows/; open the one you need:

Common Failure Modes

Moved to references/common-failure-modes.md to keep this file short; read it when this part is needed.

Best Practices

  • Store all inputs as AiiDA nodes before submission. submit(builder) stores unstored nodes automatically, but calling .store() explicitly gives you the PK/UUID immediately for cross-referencing with external systems.
  • Use Groups to organise every campaign. Create one Group per campaign (Group.objects.get_or_create(label=campaign_label)) and add every submitted process and its input structure. This enables verdi process list --group, bulk QueryBuilder filtering by group, and clean bulk export. Never rely solely on label-prefix matching.
  • Always use WorkChains, not raw CalcJobs, in production. PwBaseWorkChain, Cp2kBaseWorkChain, and their equivalents implement error handling, restart logic, and exit codes that raw CalcJobs lack. A raw CalcJob that hits a wall-time limit leaves a FAILED node with no attempt at recovery.
  • Pin aiida-core and plugin versions per project. Database schemas and plugin APIs change across minor versions. Record the pinned versions in requirements.txt, commit it alongside the campaign setup scripts, and run verdi storage migrate on a database copy before any upgrade.
  • Set clean_workdir=True in production campaigns. AiiDA stores the files it needs in its repository automatically during retrieval. The remote HPC scratch directory can and should be deleted after that. Without this, scratch quotas fill up silently over long campaigns.
  • Tag every submitted process with extras. wc.set_extra("formula", formula) and wc.set_extra("campaign", campaign_label) enable QueryBuilder filtering and verdi process list grouping. Label strings alone are fragile; extras are queryable fields.
  • Use aiida-pseudo for all pseudopotential management. Pseudopotentials installed via aiida-pseudo install sssp are stored as UpfData nodes in the AiiDA repository with checksums, are included in exported archives, and can be imported on other machines with matching UUIDs. Manually placed UPF files referenced by local path break portability and reproducibility.
  • Export provenance archives early and regularly. verdi archive export -O campaign.aiida -G <group> creates a portable, importable snapshot. Export after each milestone (cutoff convergence, relaxations complete, phonons complete). A database corruption between exports loses only the most recent work. Archives are immutable; they cannot be accidentally edited.
  • Validate computer configuration after any network or HPC change. verdi computer test <computer> verifies SSH connectivity, shell availability, scheduler command parsing, and working directory access. Run it whenever HPC hostnames, SSH keys, module paths, or scheduler configurations change. A failing computer test explains most SUBMIT_FAILED errors.

Integration with Other Skills

  • high-throughput-dft: AiiDA is an alternative to atomate2/jobflow for HT-DFT orchestration, with stronger provenance guarantees and better cross-institutional portability. The convergence tier taxonomy (screening vs. standard vs. high-accuracy), quality flag methodology, and deduplication strategy from that skill apply equally to AiiDA-managed campaigns. AiiDA and atomate2 can coexist in the same research group: use atomate2 for rapid internal campaigns, AiiDA for publication-grade or cross-institutional work.
  • quantum-espresso: aiida-quantumespresso wraps QE's pw.x, ph.x, nscf.x, and pp.x as AiiDA CalcJobs. The ecutwfc, ecutrho, degauss, ibrav=0, and pseudopotential selection rules from the QE skill are the domain knowledge that must be encoded in the Dict parameters passed to PwCalculation and PwBaseWorkChain. The SSSP pseudopotential family installed via aiida-pseudo corresponds directly to the SSSP library described in that skill.
  • cp2k-workflow: aiida-cp2k provides Cp2kCalculation and Cp2kBaseWorkChain for running CP2K DFT and AIMD workflows under AiiDA provenance. CP2K input files are constructed as nested Python dicts (converted to &SECTION format by the plugin). CP2K is the preferred AiiDA backend for large-scale AIMD and condensed-phase simulations. The CUTOFF, REL_CUTOFF, and OT/diagonalization settings from the CP2K skill must be encoded as explicit Dict nodes, not embedded as plugin defaults.
  • vasp-workflow: aiida-vasp provides VASP workflow support under AiiDA. VASP's proprietary license means POTCAR files must be installed manually on each Computer and are not redistributable via aiida-pseudo. pymatgen VaspInputSet classes can generate INCAR and KPOINTS, which are then wrapped as Dict nodes before submission. [EXPERT REVIEW NEEDED: verify aiida-vasp compatibility with AiiDA 2.5 and VASP 6.4]
  • dft-convergence: The cutoff scan, k-mesh convergence, and smearing checks from the convergence skill are the domain foundation for the AiiDA parameter sweeps in Workflow 4. AiiDA preserves the full convergence series as a set of linked WorkChainNodes sharing a common StructureData input node, making the chosen production parameters auditable in the provenance graph without requiring a separate convergence log.
  • dft-dataset-generation: AiiDA provides a provenance-tracked alternative to ad hoc SLURM arrays for DFT training data generation. Workflow 5 above extracts energy, forces, and stresses from PwCalculation output nodes and writes them to extxyz with the CalcJob UUID as a per-frame identifier, satisfying the dataset-level provenance requirements described in that skill.
  • materials-databases-access: Starting structures for AiiDA campaigns are typically retrieved from the Materials Project via mp-api or from AFLOW, COD, or ICSD. The retrieved pymatgen Structure objects are immediately converted to StructureData nodes and stored before submission, establishing a direct provenance link from the external database record to the AiiDA calculation. The ICSD and COD identifiers can be stored as node extras for traceability.
  • pymatgen-analysis: StructureData.get_pymatgen() and StructureData(pymatgen=...) are the primary interfaces between AiiDA's provenance graph and pymatgen's analysis tools. Phase diagram construction via PhaseDiagram, structure symmetrization via SpacegroupAnalyzer, and convex hull stability analysis all happen on pymatgen objects extracted from AiiDA via QueryBuilder.
  • ase-framework: StructureData.get_ase() and StructureData(ase=atoms) are the low-overhead bridge for structure manipulation and extxyz export. The ASE interface is preferred over pymatgen for bulk trajectory export from AiiDA (Workflow 5) due to lower conversion overhead for large frame counts.
  • reproducibility-workflow: AiiDA's provenance graph is the strongest available reproducibility tool for DFT campaigns — the graph itself is the provenance record. The complementary layer from that skill (environment snapshots, pseudopotential checksums, DVC for exported archives) applies to the AiiDA setup: pin and record aiida-core and plugin versions in requirements.txt; checksum AiiDA archive files with SHA-256 before depositing to NOMAD; version-control verdi computer and verdi code configuration YAML files alongside campaign setup scripts.
  • scientific-visualization: BandsData nodes from PwBandsWorkChain are visualized with pymatgen's BSPlotter or via verdi data bands export --format gnuplot. Phonon dispersion from PhononWorkChain is exported as a BandsData node and plotted with phonopy. Relaxation trajectories stored as TrajectoryData nodes are exported to extxyz and visualized with OVITO.
  • mlp-validation: DFT reference data in AiiDA (energies, forces, stresses from PwCalculation or Cp2kCalculation output nodes) can be exported as extxyz test sets for MLP validation (Workflow 5). The CalcJob UUID provides a unique, traceable identifier for each frame, enabling exact traceability from an MLP validation error back to the specific DFT calculation and its full provenance chain.

Key References

  1. Pizzi, G., Cepellotti, A., Sabatini, R., Marzari, N. & Kozinsky, B. (2016). AiiDA: Automated interactive infrastructure and database for computational science. Computational Materials Science, 111, 218–230. https://doi.org/10.1016/j.commatsci.2015.09.013
  2. Huber, S. P., Zoupanos, S., Uhrin, M., Talirz, L., Kahle, L., Häuselmann, R., Gresch, D., Müller, T., Yakutovich, A. V., Andersen, C. W., Ramirez, F. F., Adorf, C. S., Gargiulo, F., Kumbhar, S., Passaro, E., Johnston, C., Merkys, A., Cepellotti, A., Mounet, N., Marzari, N., Kozinsky, B. & Pizzi, G. (2020). AiiDA 1.0, a scalable computational infrastructure for automated reproducible workflows and data provenance in computational science. Scientific Data, 7, 300. https://doi.org/10.1038/s41597-020-00638-4
  3. Uhrin, M., Huber, S. P., Yu, J., Marzari, N. & Pizzi, G. (2021). Workflows in AiiDA: Engineering a high-throughput, event-based engine for robust and modular computational workflows. Computational Materials Science, 187, 110086. https://doi.org/10.1016/j.commatsci.2020.110086
  4. Huber, S. P., Bosoni, E., Bercx, M., Bröder, J., Degomme, A., Dikan, V., Eimre, K., Flament, M., García, A., Grisafi, A., Guda, S., Hourahine, B., Huran, A. W., Illas, F., Keller, M., Laskowski, R., Lin, L., Mercado, R., Mostofi, A. A., Petretto, G., Ratcliff, L. E., Rignanese, G.-M., Schütt, O., Timrov, I., Torrent, M., Van Hove, M., Wirtz, L., Zhang, C., Normand, J., Marzari, N., Goedecker, S., Needs, R. J., Marzari, N. & Pizzi, G. (2021). Common workflows for computing material properties using different quantum engines. npj Computational Materials, 7, 291. https://doi.org/10.1038/s41524-021-00594-6
  5. Bosoni, E., Beal, L., Bercx, M., Blöchl, P., Bühlmann, M., Cappi, F., Castelli, I., Cepellotti, A., Cerqueira, T., Corsetti, F., & Pizzi, G. (2024). How to verify the precision of density-functional-theory implementations via reproducible and universal workflows. Nature Reviews Physics, 6, 45–58. https://doi.org/10.1038/s42254-023-00655-3
  6. Talirz, L., Kumbhar, S., Passaro, E., Yakutovich, A. V., Granata, V., Gargiulo, F., Borelli, M., Uhrin, M., Huber, S. P., Zoupanos, S., Adorf, C. S., Andersen, C. W., Schütt, O., Pignedoli, C. A., Passerone, D., VandeVondele, J., Schulthess, T. C., Smit, B., Pizzi, G. & Marzari, N. (2020). Materials Cloud, a platform for open computational science. Scientific Data, 7, 299. https://doi.org/10.1038/s41597-020-00637-5

示例提示词

帮我用AiiDA搭建一套可追溯的DFT计算工作流并提交到HPC集群。

所属插件

计算材料学:第一性原理与原子模拟面向计算材料研究:DFT 与第一性原理(VASP、Quantum ESPRESSO、CP2K、AiiDA)、缺陷/表面/声子/弹性常数、机器学习势函数训练与验证、分子动力学与增强采样、晶体性质图神经网络与生成模型。

安装该插件会一并获得同领域的 35 个 Skill / MCP。

同插件的其他科研 Skill

常见问题

AiiDA计算工作流管理能做什么?

Covers AiiDA, a provenance-first Python workflow engine for computational materials science: calculation input generation, remote submission. Use when orchestrating reproducible DFT/MD campaigns。

如何安装AiiDA计算工作流管理?

在科搭 Codar 左侧点击「插件」,在「组件」页签搜索「AiiDA计算工作流管理」并安装;在 Claude Code 或 Codex 中可以发送本页的安装提示词,或下载 Skill 包后解压到 ~/.claude/skills 或 ~/.codex/skills。

AiiDA计算工作流管理需要哪些权限?

该科研 Skill没有声明额外权限;执行具体操作时仍由 Agent 按自身规则请求确认。

AiiDA计算工作流管理经过官方审核吗?

是。SpeedSkills 收录的每个版本都会记录内容摘要与权限声明,当前版本已通过安全审核。