科研 Skill
学术文献获取下载
acquire-sources
Auto-downloads open-access PDFs for graded sources and writes a manual worklist for paywalled items. Use after literature-review, before ingest-source.
SKILL.md
Boundary: literature-review → acquire-sources → ingest-source
literature-review runs SEARCH — it builds the graded guide and BibTeX. It downloads nothing.
acquire-sources runs ACQUISITION — it turns the A+B graded guide into PDF files on disk, and for everything it cannot fetch automatically it writes a manual-download worklist (acquisition-todo.md). This skill owns all downloading.
ingest-source runs INTAKE — it reads an already-acquired original. It does not download, and it hard-stops rather than silently substituting a preprint/review for a missing original.
Sequence: literature-review once → acquire-sources once (re-run to reconcile) → ingest-source N times.
Acquire Sources (OA auto-download + manual worklist)
Get the actual PDFs onto disk before ingest. Two outcomes per source: either an Open-Access copy is fetched automatically into the shared library (<library>/pdf/), or the source goes onto a worklist the user clears manually (university VPN / library proxy reach far more originals than a bot can — publishers block automated downloads). The manual worklist is a first-class output, not a failure: it is the bridge that lets the human do what the agent cannot.
Announce at start: "Using acquire-sources to obtain the A+B PDFs and build a manual-download worklist for the rest."
Report counts: "Acquired N of M A+B sources. K need manual download — see acquisition-todo.md, fetch via VPN into <library>/pdf/ under the exact <bibkey>.pdf filenames, then re-run acquire-sources (or proceed to ingest the present ones)." If a condition is unmet: name it, ask for a one-line reason, write it to knowledge/_meta/gate-overrides.log, and close.
When to use
literature-reviewhas produced aliteraturguide.mdand you want the PDFs before ingest- You manually downloaded some originals via VPN and want to reconcile the worklist (re-run)
- A draft or ingest run reported a missing original and you need to acquire it
executing-research-plan's acquisition gate invokes this before ingest tasks — automatically inside a plan run, not only as a standalone user-triggered step (the skill is idempotent and reconcile-capable, so it is safe to call from the gate's resume loop)
NOT for: searching (use literature-review), reading a source into the wiki (use ingest-source), or downloading a single ad-hoc URL.
Checklist
Create a task-list item for each:
- Build the A+B worklist — read
literaturguide_pathandbibtex_path. The guide's weighted source table (section 1, columnsGrade | Autor Jahr | Kurztitel | OA/Zugang | DOI/Link) is the canonical weighting: filter on itsGradecolumn bygrade_scope(defaultA,B). For every in-scope source collect:bibkey, authors, year, short title (theKurztitelcell), grade,doi,url,oa_pdf(the per-candidate fields theliterature-scoutagent emits), and the canonical target filename<bibkey>.pdf— the bibkey IS the filename. The bibkey isautor-jahr-kurztitel, all lowercase ASCII, hyphen-separated:autor= first author's surname (umlauts →ae/oe/ue,ß→ss; Turkishı→i, Polishł→l; particles/hyphens removed),jahr= four-digit year (letter suffix after the year for clashes,finkelstein-2003b),kurztitel= the first significant title word, stopwords dropped. e.g.finkelstein-2003-low-chronology.pdf. Into<library>/pdf/— resolve the library withscripts/library.py. - Reconcile against the library first (idempotent) — scan
<library>/pdf/*.pdf. Match each worklist item by its<bibkey>.pdffilename — an exact match, no fuzzy guessing needed: that is the whole point ofbibkey == filename. A match →already-present; never re-download it. Note the library is SHARED: another project may already have acquired this source, and that counts. - Resolve a download URL for each still-missing item, in priority order:
(a)
oa_pdfcarried in the guide / scout output; (b)dao-paper-search-mcpby DOI —search_crossref/search_openalex/search_core/search_zenodo/search_arxiv→oa_pdf(OpenAlex:best_oa_location.pdf_url); (c) Unpaywall by DOI (https://api.unpaywall.org/v2/<doi>?email=<contact>→best_oa_location.url_for_pdf); (d) none found →manualwith reasonno_oa_url. - Attempt the download (curl, validated — see below) →
downloadedormanual(with reason). - Regenerate
acquisition-todo.md— only themanualitems, with the instruction header. Resolved items drop out automatically on each run. - Write
acquisition-log-<date>.json— the audit (see below). - Append to
knowledge/_meta/log.md— oneacquireentry. - Re-index the search —
python scripts/bib-search.py index. Incremental (seconds); this is the moment new PDFs become searchable, and the index run is what surfaces scans with no text layer (statuslists them) while the acquisition context is still loaded. Skipping this leaves the drift for the session-start check to find later — acquiring is the action that caused it, so acquiring pays for it. - Gate & report per the SOFT-GATE.
Download mechanism — concrete and validated (the core)
WebFetch /
fetch_urlreturn extracted markdown, not a binary file — they cannot save a PDF. The real download is Bashcurlinto a temp file, followed by validation. The #1 failure mode is a saved HTML "Access Denied" / login / Cloudflare page that looks like a successful 200 — it must NEVER be mistaken for a PDF.
URL="…"
TARGET="$LIBRARY/pdf/<bibkey>.pdf" # e.g. finkelstein-2003-low-chronology.pdf; $LIBRARY from scripts/library.py
TMP="$(mktemp -t acq).pdf"
# Capture curl's OWN exit status (network errors → curl_error). The -w format
# ends in \n so the later `read` gets a terminated line and does not falsely
# report failure.
META=$(curl -sS -L \
-A "Mozilla/5.0 (research-superpowers acquire-sources; mailto:<contact>)" \
--max-time 90 --retry 2 \
-w '%{http_code} %{content_type}\n' \
-o "$TMP" "$URL") || { rm -f "$TMP"; echo "manual:curl_error"; }
read -r CODE CTYPE <<< "$META" # e.g. CODE=200 CTYPE=application/pdf
Accept the file as downloaded only if ALL hold; otherwise classify manual and rm "$TMP":
- HTTP 200 (
$CODE== 200). - Content-Type contains
application/pdf($CTYPE, cross-checked withfile --mime-type "$TMP").text/html→ paywall/login →manual(not_pdf_content_type). - Magic bytes — first 5 bytes are
%PDF-(head -c5 "$TMP"). Catches HTML served with a wrong content-type. - Non-trivial size —
> ~10 KB(wc -c < "$TMP"). A 2 KB "file" is an error page (too_small). - Not an HTML error/login page — reject if the head starts with
<!DOC/<html/{, or contains a<title>with "Access Denied" / "Just a moment" (Cloudflare) / "Sign in" / "Captcha" (html_login_page/cloudflare_block).
On success: mv "$TMP" "$TARGET", classify downloaded. Record the failure reason on every manual: http_4xx | not_pdf_content_type | html_login_page | cloudflare_block | too_small | no_oa_url | curl_error.
Source-class routing (decide before burning a curl attempt):
| OA-friendly → attempt curl | Route straight to manual (do not attempt) |
|---|---|
| arXiv, Zenodo, CORE | Elsevier / Springer / Wiley / Taylor&Francis / SAGE paywalls |
OpenAlex best_oa_location.pdf_url | JSTOR; De Gruyter / Brill (usually paywalled) |
| Institutional repositories / DOAJ / OAPEN | Anything behind Cloudflare ("Just a moment") |
Unpaywall-resolved url_for_pdf | Publisher landing pages with no OA PDF; Google Books |
The validation step is the real safety net: an over-optimistic attempt that returns HTML is still caught and downgraded to manual.
MCP soft-preference (recommended). If dao-paper-search-mcp (see docs/recommended-mcps.md) is available, resolve OA links via its oa_pdf field and feed audit.source_class into routing — aggregator / suspect → manual, never auto-download. Otherwise use the guide's oa_pdf, then Unpaywall.
acquisition-todo.md — the manual-download worklist
Regenerated on every run (resolved items disappear). Layout:
# Source acquisition — manual downloads needed
These A+B sources could not be auto-downloaded (paywalled, bot-blocked, or no
Open-Access copy). Download each **original** PDF via your university VPN /
library proxy, save it **into `<library>/pdf/`** under
the **exact** "Save as" filename below, then **re-run acquire-sources** to
reconcile (resolved rows disappear) — or run `ingest-source` on the ones
already present.
Do NOT substitute a preprint, prior version, or book review for the original —
`ingest-source` hard-stops on a missing original.
Licensed PDFs (UB/VPN): check whether your library's licence allows processing
by the AI provider you use; if not, ingest them with a local or
institution-hosted model.
Last updated: <YYYY-MM-DD> · <K> of <M> A+B sources still needed.
| ☐ | Citekey | Author Year | Short title | Grade | DOI / landing | Best download URL(s) | Save as (exact filename) |
|---|---------|-------------|-------------|-------|---------------|----------------------|--------------------------|
| [ ] | finkelstein-2003 | Finkelstein 2003 | Low Chronology Revisited | A | [10.1179/…](https://doi.org/10.1179/…) | <links or "—"> | `finkelstein-2003-low-chronology.pdf` |
Licence note. Where the resolver returned a licence (Unpaywall / OpenAlex
license, e.g. cc-by, cc-by-nc-nd) record it in the audit JSON per item. A
PDF obtained through a university licence (◐ UB) or a publisher platform may come
with terms that restrict processing by AI services; many licence agreements now
say so explicitly. That is the user's call, not this skill's — but make it
visible: the worklist header carries one line, "Licensed PDFs (UB/VPN): check
whether your library's licence allows processing by the AI provider you use; if
not, ingest them with a local or institution-hosted model." CC-BY / CC-BY-SA
and public-domain items are unproblematic.
The Citekey and Save as columns are load-bearing: they tie the manually-downloaded file to its BibTeX key and to the filename ingest-source expects. DOI / landing gives the canonical resolver; Best download URL(s) lists any concrete-but-unfetchable PDF candidate worth a browser try.
acquisition-log-<date>.json — the audit (date-stamped, never clobbered)
Mirrors the audit-log-<date>.json convention of literature-review:
{
"skill": "acquire-sources",
"date": "2026-06-23",
"literaturguide": "input/bibliography/literaturguide.md",
"grade_scope": ["A", "B"],
"totals": { "worklist": 18, "downloaded": 9, "already_present": 2, "manual": 6, "skipped": 1 },
"items": [
{
"bibkey": "finkelstein-2003-low-chronology", "grade": "A", "doi": "10.1179/…",
"resolved_url": "https://…", "url_source": "unpaywall|oa_pdf|crossref|openalex|core|none",
"outcome": "downloaded|already-present|manual|skipped",
"reason": "html_login_page|not_pdf_content_type|cloudflare_block|too_small|no_oa_url|null",
"saved_as": "finkelstein-2003-low-chronology.pdf",
"validated": { "http": 200, "content_type": "application/pdf", "magic_pdf": true, "bytes": 842113 }
}
]
}
Log entry
Append to knowledge/_meta/log.md (the heading-prefixed form used by ingest/lint):
## [2026-06-23] acquire | A+B sources
- Downloaded 9, already present 2, manual 6, skipped 1 (of 18 A+B)
- Manual worklist: [[acquisition-todo]] (6 items)
- Audit: input/bibliography/acquisition-log-2026-06-23.json
Reconcile / re-run mode (idempotent — the loop)
Re-running is safe and is the intended loop:
- Step 2 rescans
<library>/pdf/*.pdf. Any item now present (the user fetched it via VPN, or a flaky OA link now works) →already-present, removed from the regeneratedacquisition-todo.md. - Still-missing items stay; their URLs are re-resolved (an OA copy may have appeared since).
downloaded/already-presentfiles are never re-fetched.- The audit JSON is written fresh per run; its date-stamped filename means no clobber.
- Report the delta: "Since last run: 4 manual items now present, 2 still missing." The loop is: download some → re-run → ingest the present ones.
Process Flow
digraph acquire {
"Build A+B worklist" [shape=box];
"Reconcile against disk" [shape=box];
"Already present?" [shape=diamond];
"Resolve OA URL" [shape=box];
"OA URL found?" [shape=diamond];
"Publisher paywall host?" [shape=diamond];
"curl + validate" [shape=box];
"Valid PDF?" [shape=diamond];
"downloaded" [shape=box];
"manual (with reason)" [shape=box];
"Write acquisition-todo.md" [shape=box];
"Write audit JSON" [shape=box];
"Append log" [shape=box];
"Gate & report" [shape=doublecircle];
"Build A+B worklist" -> "Reconcile against disk";
"Reconcile against disk" -> "Already present?";
"Already present?" -> "Write acquisition-todo.md" [label="yes (skip download)"];
"Already present?" -> "Resolve OA URL" [label="no"];
"Resolve OA URL" -> "OA URL found?";
"OA URL found?" -> "manual (with reason)" [label="no — no_oa_url"];
"OA URL found?" -> "Publisher paywall host?" [label="yes"];
"Publisher paywall host?" -> "manual (with reason)" [label="yes"];
"Publisher paywall host?" -> "curl + validate" [label="no"];
"curl + validate" -> "Valid PDF?";
"Valid PDF?" -> "downloaded" [label="yes"];
"Valid PDF?" -> "manual (with reason)" [label="no"];
"downloaded" -> "Write acquisition-todo.md";
"manual (with reason)" -> "Write acquisition-todo.md";
"Write acquisition-todo.md" -> "Write audit JSON";
"Write audit JSON" -> "Append log";
"Append log" -> "Gate & report";
}
Subagent Dispatch (recommended for large sets)
For a worklist of ≥ ~8 items, dispatch the source-acquirer subagent (see agents/source-acquirer.md). Downloads are noisy — curl progress, validation failures, HTML dumps from blocked pages — exactly the context pollution a subagent isolates (the same rationale literature-scout uses for search noise and source-ingester for per-PDF reading). The subagent runs the resolve+curl+validate batch (steps 3–4) and returns a compact status table; the parent never sees the raw chatter. The decisions — grade_scope, confirming the manual list, the gate report — stay in the main conversation. Small sets run inline.
Red Flags
| Thought | Reality |
|---|---|
| "WebFetch gave me the PDF text, so it's downloaded" | No — WebFetch returns markdown, not a file. Only a validated curl save counts. |
| "curl returned 200, so it's a PDF" | 200 + an HTML login page is the commonest failure. Check content-type AND %PDF- magic bytes AND size. |
| "The preprint is basically the same — I'll save it as the original" | No — wrong pagination, possibly different claims. The original goes on the manual worklist; substitution is ingest-source's consent-gated decision, not yours. |
| "The A-grade ones are paywalled, I'll skip them" | A-grade is must-read. Paywalled ≠ skip — it goes on the manual worklist for the user's VPN. |
| "URLs in the guide are enough" | Link rot is real. The whole point of this phase is files on disk before ingest. |
| "I'll just re-download everything each run" | No — reconcile against disk first; already-present files are never re-fetched. |
| "An empty acquisition-todo.md means I failed" | The opposite — empty means every A+B source is on disk. The worklist is a tool, not a defect. |
Key Principles
- Files on disk before ingest — the manual worklist is a first-class output, not a failure.
- Validate every download — content-type + magic bytes + size + not-HTML. A saved error page is never a source.
- Idempotent — reconcile against disk first; re-run any number of times.
- Exact filenames tie manual downloads to ingest —
<bibkey>.pdfin<library>/pdf/. The filename IS the citekey, soingest-sourcefinds the original without guessing. - OA-first, paywall-routed — attempt the open repositories, route publisher paywalls straight to the human.
- Transparency — the audit JSON records every resolution and every failure reason.
示例提示词
帮我把文献筛选出的A类来源批量下载成PDF并列出需要人工获取的清单。
所属插件
安装该插件会一并获得同领域的 62 个 Skill / MCP。
同插件的其他科研 Skill
- 单篇文献入库管理Adds one verified PDF to a shared bibliography library, computing its citation key and keywords. Use to register a single document outside the full workflow.
- 数字人文计算方法Surveys computational DH methods: text mining, NLP, corpus linguistics, archive standards, GIS, network analysis, and OCR. Use for an overview of available DH tools.
- 考古地理信息系统分析Runs archaeological GIS analysis with GeoPandas: catchment analysis, viewshed, kernel density, spatial statistics. Use for mapping archaeological site locations.
- 论证结构提取可视化Extracts and visualizes argument structure: Toulmin reconstruction, support/attack graphs, circularity detection. Use to diagram an argument’s logical structure.
- 考古遗物类型学分析Performs artifact analysis: typological classification, ceramic attribute coding, lithic reduction analysis. Use when classifying excavated artifacts statistically.
- 引文与出处核查Checks citations, quotations, page numbers, and bibliography entries for accuracy and source fit. Use before submission to verify source-supported claims.
- 研究选题构思设计Explores research intent, field state, methodology, and ethics, then writes an approved design doc. Use before starting a new research question or chapter.
- 术语译名一致性检查Audits a manuscript for terms, translations, abbreviations, and names whose forms must stay stable. Use when terminology drift could undermine an argument.
- 计算语用学分析Runs computational pragmatics: speech act classification, implicature detection, discourse coherence, politeness analysis. Use for language-use-and-meaning studies.
- 文献综述学术对话构建Turns verified sources into a structured scholarly conversation among positions, not a chronological summary. Use when synthesizing literature into a debate narrative.
- 语料库语言学分析Performs corpus analysis with NLTK/spaCy: collocation, keyword analysis, KWIC concordance, word2vec, diachronic trends. Use for quantitative text-corpus research.
- 数字档案馆资源检索Queries digital archive REST APIs: Europeana, Chronicling America, Internet Archive, HathiTrust, DPLA, OAI-PMH. Use when searching or harvesting heritage metadata.
- 学术手稿章节撰写Drafts a chapter or article section by section from synthesized wiki content, agreeing each section’s argument first. Use to turn stable research synthesis into prose.
- 研究项目漂移状态检测Checks a shared source library and registered projects for drift: un-indexed PDFs, unmerged keywords, bibkey collisions. Use to audit a multi-project research setup.
- EMI课堂语料转写分析Segments and annotates classroom transcripts, coding language choice and discourse function with anonymised output. Use to quantify classroom talk or translanguaging.
- 人工智能伦理分析Analyzes AI ethics: algorithmic fairness metrics, bias auditing, value alignment, moral philosophy reasoning. Use to evaluate an AI system’s ethical dimensions.
- 研究计划任务执行Executes a ready research plan task by task, gating acquisition before ingest and dispatching analysis. Use after writing-research-plan to run the planned work.
- 研究项目结项清单Runs the final completion checklist: rendering, bibliography, wiki, data, reproducibility, archiving. Use when a manuscript is finalized and ready to publish.
- 形式逻辑自动推理Applies formal logic tools: Z3 SMT solver, modal logic S5, SAT encoding, Lean4 proof verification. Use to formalize or mechanically check a logical argument.
- 研究问题凝练构建Turns a broad topic or intuition into a precise, contestable, researchable question. Use at the start of a project when no defensible question yet exists.
- 历史地理信息系统分析Performs historical GIS work: georeferencing old maps, digitizing boundaries, temporal territory overlays, population interpolation. Use for historical spatial analysis.
- 历史语言学比较分析Analyzes historical linguistics in Python: sound correspondences, cognate detection, phylogenetic trees, Swadesh lists. Use when reconstructing language relationships.
- 历史社会网络分析Builds historical network analyses: correspondence networks, prosopography, temporal community detection, Gephi export. Use to map relationships among historical actors.
- 历史文献OCR识别Transcribes printed and handwritten historical documents with Tesseract/Kraken OCR, confidence filtering, spell correction. Use for digitizing scanned historical text.
常见问题
学术文献获取下载能做什么?
Auto-downloads open-access PDFs for graded sources and writes a manual worklist for paywalled items. Use after literature-review, before ingest-source。
如何安装学术文献获取下载?
在科搭 Codar 左侧点击「插件」,在「组件」页签搜索「学术文献获取下载」并安装;在 Claude Code 或 Codex 中可以发送本页的安装提示词,或下载 Skill 包后解压到 ~/.claude/skills 或 ~/.codex/skills。
学术文献获取下载需要哪些权限?
该科研 Skill没有声明额外权限;执行具体操作时仍由 Agent 按自身规则请求确认。
学术文献获取下载经过官方审核吗?
是。SpeedSkills 收录的每个版本都会记录内容摘要与权限声明,当前版本已通过安全审核。