科研 Skill
DOI全文批量获取
fulltext-retrieval
Batch-downloads open-access PDFs for a list of DOIs via Unpaywall, PMC and OpenAlex. Use when full texts are needed for a screening set such as a meta-analysis.
SKILL.md
Fulltext Retrieval Skill
Batch download open-access full-text PDFs from a DOI list using legitimate OA APIs only.
Paywalled articles fail by design and are listed in manual_needed.txt for institutional access
or ILL; never work around a paywall or publisher access control.
Pipeline
DOI → arXiv (10.48550/arXiv.* DOIs) → Unpaywall → PMC (Europe PMC / OA FTP / web) → OpenAlex → Crossref → landing page
Each DOI goes through these sources in order until a valid PDF (≥10 KB, %PDF- header) is found.
arXiv DOIs (10.48550/arXiv.2401.01234, version suffixes, old-style hep-th/9901001, or a bare
arXiv: id) resolve directly to the arXiv PDF first.
Run
Requires Python 3.10+ (stdlib only) and a contact email, which Unpaywall's Terms of Service require. The script paces its requests (0.3–0.5 s delays) for the APIs' rate limits.
python "fetch_oa.py" dois.txt --output pdfs/ --email your@email.com
# Verbose mode for debugging (per-DOI source trace)
python "fetch_oa.py" dois.txt -o pdfs/ -e your@email.com --verbose
Input formats:
- Plain text — one DOI per line.
- TSV / CSV with header, or a Markdown pipe table — must contain a
DOIcolumn; optionalPMID,Title, andFirstAuthor(surname or full name) columns.
A PMID makes the PMC lookup more reliable (PMID → PMCID conversion). Supply Title where
available: a DOI-only worklist can download a PDF but cannot establish title agreement.
FirstAuthor is optional additional evidence.
Output
- PDFs saved as
{DOI_safe}.pdf(slashes replaced with underscores). pdfs/retrieval_report.json— structured per-DOI report (below); override with--report PATH.<output>/manual_needed.txt— DOIs that could not be retrieved via OA; when a PMCID was resolved, the line also carries it and the PubMed Central article URL to open in a browser.- Summary with arXiv/OA/PMC/fail/skip counts.
Retrieval report (--report)
Every run writes the report (default <output>/retrieval_report.json), schema 2:
{
"schema_version": 2,
"generated_by": "fetch_oa.py",
"counts": {"total": 4, "retrieved": 3, "not_retrieved": 1, "title_mismatch": 1,
"source_identity": {"consistent": 1, "conflict": 1, "unresolved": 1, "unavailable": 1}},
"items": [
{"doi": "10.1000/synthetic.example", "pmid": "", "title": "Example title",
"first_author": "", "status": "oa", "source": "unpaywall",
"file": "10.1000_synthetic.example.pdf", "size_bytes": 482113, "page_count": 9,
"file_sha256": "<SHA-256 of the downloaded file>", "title_match": "match",
"source_identity": {"status": "consistent", "reason": "title_and_identifier_agree",
"text_scope": "first_page_front_matter", "title_match": "match",
"doi_match": "match", "observed_identifiers": ["10.1000/synthetic.example"],
"first_author_match": "unavailable"}}
]
}
status (arxiv | oa | pmc | skip | fail), source, and counts.retrieved describe the
resolver result, including existing files (skip). They do not count identity-verified
papers. No PDF is automatically deleted or rejected. page_count comes from Poppler's
pdfinfo (null without it) and is recorded, not judged — a 3-page "article" or a 4-page "book"
is worth opening.
source_identity.status | Meaning / action |
|---|---|
consistent | Complete normalized title and a compatible DOI/arXiv identifier occur in the bounded first-page front matter, with no supplement / preface / table-of-contents heading and no retraction / erratum / correction / corrigendum / expression-of-concern heading there; an optional supplied author must also match. Evidence agrees, but this is not independent source verification or claim validation. |
conflict | Both the title and observed identifier differ. Inspect the PDF and requested record. |
unresolved | Evidence is incomplete or ambiguous: title-only, DOI-only, missing author, multiple identifiers, a matching title with another DOI/version, or a supplement / preface / table-of-contents file that names the work without being it (supplement_or_front_matter). A retraction notice, erratum, correction, corrigendum or expression of concern whose heading line sits in that area is likewise unresolved (correction_or_retraction_notice). Inspect before using as evidence. |
unavailable | No usable extracted text, Poppler unavailable, no output PDF, or the PDF changed during assessment. No current identity assessment was possible. |
Evidence is limited to the first page before a recognized abstract/body/reference heading (at
most 40 lines / 4,000 characters), so a title cited in the body or references does not count. A
title_match of match needs the complete normalized title on up to six consecutive lines;
partial overlap is unavailable and low overlap an advisory mismatch. Cover sheets, unusual
reading order, short or changed titles and DOI footers outside that area can stay unresolved.
PDF metadata and the filename alone are not identity evidence. Explicit arXiv versions must
agree; a preprint/published-version DOI difference needs review, not automatic rejection.
Downstream reports must preserve source_identity and file_sha256, keep unresolved items
visible, and check the hash still identifies the file being used. Older reports without identity
evidence remain unassessed; do not infer identity from retrieved or title_match=match.
Full-text conversion does not resolve an identity warning.
Attach PDFs into Zotero ("Find Available PDF")
For paywalled-but-licensed papers the OA resolvers miss, references/find_available_pdf.js is a
user-run snippet for Zotero's Tools → Developer → Run JavaScript (no-code equivalent:
right-click → "Find Available PDF"). It triggers Zotero's own addAvailablePDF /
addAvailablePDFs, so it reuses the user's OpenURL resolver / institutional proxy config;
no credentials, proxy hosts, or institutional identifiers are hard-coded or leave the Zotero
client. It is user-initiated and depends on the live Zotero session, so record its results
manually — they are not reproducible CI evidence. /lit-sync Phase 2.7 orchestrates both routes
and reconciles them in a report.
PDF → Markdown Conversion (Optional)
Convert downloaded PDFs to Markdown when the same papers will be read repeatedly (data extraction from k≥5 studies, a meta-analysis pipeline); for one-pass screening, read the PDF directly.
# Install (one-time)
pip install pymupdf4llm
# Convert all PDFs in a directory (.md files land alongside the .pdf files)
python "pdf_to_md.py" pdfs/ -v
# Custom output directory
python "pdf_to_md.py" pdfs/ -o markdown/
# First 10 pages only (useful for long supplements)
python "pdf_to_md.py" pdfs/ --pages 0-9
# Overwrite existing conversions
python "pdf_to_md.py" pdfs/ --force
Images are skipped, so figures survive only as caption text. Scanned-only PDFs (no text layer)
convert poorly. pdf_to_md.py needs pymupdf4llm
(AGPL-3.0), an optional dependency; fetch_oa.py stays stdlib-only.
示例提示词
帮我批量下载这份荟萃分析文献清单对应DOI的开放获取全文。
所属插件
安装该插件会一并获得同领域的 58 个 Skill / MCP。
同插件的其他科研 Skill
- 医学AI论文检索优化Optimizes a medical AI paper’s title, abstract and metadata for AI search engines. Use when a paper should be findable and citable by RAG tools and AI search.
- 临床数据统计分析与建模Generates reproducible Python or R code for clinical statistics and tables. Use when collected data needs Table 1, regression, survival or agreement analysis.
- 作者发表记录与策略分析Analyzes a PubMed author record to classify study types and authorship patterns. Use when planning a publication strategy from a researcher’s track record.
- 批量队列暴露结局分析Generates one cohort-analysis script per exposure-outcome pair from one template. Use when a validated method must run across many variable combinations at once.
- 样本量估算与检验效能Walks a decision tree to the right test and computes sample size with IRB text. Use when planning how many patients a study needs before data collection.
- 因果推断方法设计Runs DAG-based, instrumental-variable, difference-in-differences or RD analyses. Use when an association in observational data must support a causal claim.
- 报告规范逐条核查Audits a manuscript item by item against 49 reporting and risk-of-bias checklists. Use when checking a draft against STROBE, CONSORT, TRIPOD or similar guidelines.
- 临床数据清洗与画像Profiles a clinical dataset and generates cleaning code in staged, approved steps. Use when a CSV or Excel dataset needs missing-value and outlier checks before analysis.
- 跨国队列数据协调对比Harmonizes variables and runs parallel weighted analyses across national surveys like KNHANES and NHANES. Use when comparing one exposure-outcome link across countries.
- 变量定义与截断值溯源Maps exposure, outcome and covariate definitions to a cited guideline and DB columns. Use when variable cutoffs need a citable basis before the study protocol.
- 本地化临床数据脱敏Runs a local, network-free script that detects and removes identifiers from clinical data. Use when data may contain PHI and must be de-identified before LLM analysis.
- AI对标专家基准研究设计Plans the arms, rubrics, calibration probes and reviewer panel for an AI-vs-expert benchmark. Use before data collection when designing an AI-vs-human rating study.
- 放射影像AI研究设计审查Reviews a study’s analysis unit, cohort logic, leakage risk and reporting-guideline fit. Use when checking a clinical or medical-AI study design before drafting.
- 传染病仓室模型拟合Builds and fits SIR, SEIR and SEIHR compartmental models with Rt estimation. Use when an outbreak or epidemic’s spread needs mathematical modeling in Python.
- ICMJE利益冲突声明填写Clones a pre-filled synthetic ICMJE COI form, replacing only date, name and title. Use when each author on a submission needs a disclosure form filled quickly.
- 机构Word表单内容填充Fills an institutional Word form, such as an IRB protocol, without breaking its layout. Use when drafted content must be rendered into a CJK-aware .docx template.
- 队列数据库选题挖掘Profiles a longitudinal cohort database and ranks research-topic proposals with gap evidence. Use when looking for a question an available cohort or registry can answer.
- 投稿期刊匹配推荐Matches a manuscript abstract against curated journal profiles and ranks candidates. Use when choosing where to submit, or re-targeting after a rejection.
- 数据集编码本自动生成Profiles every variable in a tabular dataset into a structured, citable codebook. Use when a CSV, Excel or Stata dataset needs a data dictionary before analysis.
- 全球卫生数据获取分析Accesses and analyzes global health data from WHO GHO, IHME GBD and Our World in Data. Use when DALYs, age-standardized rates or choropleth maps are needed.
- 医学AI项目基金申请书撰写Structures a grant proposal’s significance, innovation, approach, aims and milestones. Use when drafting a research funding proposal, including a Korean grant plan.
- 卫生经济学评价建模Builds Markov cohort models with QALYs, PSA and cost-effectiveness planes. Use when a health intervention needs a formal cost-effectiveness evaluation.
- AI写作痕迹识别改写Scans a manuscript for AI writing patterns and rewrites flagged passages carefully. Use when a draft or response letter reads as obviously AI-written.
- 医学影像数据集画像与预处理Profiles a DICOM or NIfTI dataset’s spacing, intensity and label integrity before modeling. Use when preparing an imaging dataset and auditing it for leakage.
常见问题
DOI全文批量获取能做什么?
Batch-downloads open-access PDFs for a list of DOIs via Unpaywall, PMC and OpenAlex. Use when full texts are needed for a screening set such as a meta-analysis。
如何安装DOI全文批量获取?
在科搭 Codar 左侧点击「插件」,在「组件」页签搜索「DOI全文批量获取」并安装;在 Claude Code 或 Codex 中可以发送本页的安装提示词,或下载 Skill 包后解压到 ~/.claude/skills 或 ~/.codex/skills。
DOI全文批量获取需要哪些权限?
该科研 Skill没有声明额外权限;执行具体操作时仍由 Agent 按自身规则请求确认。
DOI全文批量获取经过官方审核吗?
是。SpeedSkills 收录的每个版本都会记录内容摘要与权限声明,当前版本已通过安全审核。