科研 Skill

文本数据定量分析

alterlab-text-as-data

Analyzes text as social-science data with topic modeling, embeddings, and validated dictionary or classification methods. Use for computational text analysis beyond a keyword count.

发布者 Codar安全审核通过v0.1.0更新于 2026-11-18

SKILL.md

Text-as-Data — Match the Method to the Inferential Goal

Skill type: ANALYSIS MODULE. Turns a corpus into measurements. The discipline is choosing by goal — discovery (what themes exist?), measurement (how much of concept X?), or prediction (label new documents) — and validating that a topic solution is reliable, not a single lucky stochastic run.

Core Mission

PICK BY GOAL: DISCOVERY vs MEASUREMENT vs PREDICTION. THEN VALIDATE THE TOPICS — ONE RUN IS NOT A RESULT.

When to Use This Skill

  • "Run topic modeling on my corpus (BERTopic / LDA)."
  • "Measure how much each document expresses concept X (dictionary/lexicon)."
  • "Embed my documents and cluster / compare them."
  • "Classify these texts into categories."

Does NOT Trigger

The request is really about…Route toWhy not this skill
Training / fine-tuning a transformer modelalterlab-transformersModel training, not corpus measurement.
Humanities close-reading / annotation of textsalterlab-digital-humanitiesInterpretive, not quantitative text-as-data.
Whether a text method fits the question at allalterlab-ssci-design-gateDesign routing, upstream.
Plain tabular statisticsalterlab-statistical-analysisNo text.

Hand-off rule. When the request matches a row above, open your answer by naming that skill by its exact id (for example alterlab-transformers) and saying in one line why it fits better, then stop applying this skill's procedure. Name it even when you also need to ask for missing input. If only part of the request is out of scope, name the skill for that part and do only the in-scope part. A brief general pointer is fine; the named skill does the work.

Running the code. In Claude Code, run analysis code in a throwaway environment instead of installing packages: uv run --python ">=3.11" --with "bertopic>=0.16" --with "sentence-transformers>=2.2" python topics.py. This needs no install approval and leaves the user's environment untouched. If uv or the network is unavailable, say so and hand over the script; never report results you did not compute.

Method by goal (verified stack, pinned)

GoalMethodVerified call
Discovery (emergent themes, contextual)BERTopic (v0.17)from bertopic import BERTopic; topics, probs = BERTopic().fit_transform(docs); get_topic_info(), get_topic(0). Plug embeddings via embedding_model=SentenceTransformer("all-MiniLM-L6-v2"), a vectorizer_model=CountVectorizer(min_df=10).
Discovery (bag-of-words, classic)LDA / NMFsklearn: LatentDirichletAllocation(n_components=k).fit(CountVectorizer().fit_transform(docs)); or gensim LdaModel(corpus, num_topics=k, id2word=dictionary).
Topic qualitycoherencegensim CoherenceModel(model=lda, texts=tok, dictionary=d, coherence="c_v").get_coherence().
Measurement (how much of concept X)dictionary / lexiconcount validated lexicon terms; report reliability and validate against hand-coding.
Embeddings / similaritysentence-transformers (v6)SentenceTransformer("all-MiniLM-L6-v2").encode(texts) → cluster / cosine-compare.
Prediction (label documents)supervisedTfidfVectorizer() → a sklearn classifier; report held-out F1, not in-sample fit.
Linguistic features (POS, entities)spaCy (v3)nlp = spacy.load("en_core_web_sm"); doc = nlp(text).

Method-selection helper (stdlib): scripts/text_method_router.py. Full patterns, preprocessing, and the topic-reliability procedure: references/text_stack.md.

The reliability discipline (topic models are stochastic)

LDA (and, to a lesser degree, BERTopic via UMAP) gives different topics on different runs. A single run is not a result. Required:

  1. Replicate across seeds/runs; align topics across runs (e.g. Hungarian matching) and score stability (top-word overlap, RBO); report a prototype (most-representative) solution.
  2. Coherence, not just perplexity — choose K by c_v coherence and human readability, not log-likelihood alone.
  3. Determinism where possible — fix random_state (sklearn LDA) and UMAP's random_state (BERTopic) for reproducibility; still report stability across different seeds.
  4. Validate measurement — a dictionary/topic used as a measure of a concept needs validation against human coding (precision/recall or correlation), exactly like any other instrument.

Output Template

GOAL:         discovery | measurement | prediction
METHOD:       <BERTopic / LDA / dictionary / embeddings / supervised> + why it matches the goal
MODEL:        <call + K selection by coherence; embedding/vectorizer choices>
RELIABILITY:  <seeds run; topic stability; coherence c_v; determinism settings>
VALIDATION:   <if used as a measure: agreement with human coding>
CLAIM SCOPE:  descriptive/measurement of the corpus; generalization scoped to the corpus/sampling

References

  • references/text_stack.md — BERTopic/LDA/gensim/spaCy/embeddings patterns, preprocessing, topic-reliability procedure.
  • scripts/text_method_router.py — stdlib router from goal + corpus features to the right method.

Part of the AlterLab Academic Skills suite.

示例提示词

我想对这批社交媒体文本做主题建模,区分不同话语主题。

所属插件

社会学、政治学与城市研究面向社会科学研究:问卷与访谈设计、质性编码、文本即数据、社会网络、ABM、比较政治与选举数据、RCT 与 DID、城市与区域分析。

安装该插件会一并获得同领域的 58 个 Skill / MCP。

同插件的其他科研 Skill

常见问题

文本数据定量分析能做什么?

Analyzes text as social-science data with topic modeling, embeddings, and validated dictionary or classification methods. Use for computational text analysis beyond a keyword count。

如何安装文本数据定量分析?

在科搭 Codar 左侧点击「插件」,在「组件」页签搜索「文本数据定量分析」并安装;在 Claude Code 或 Codex 中可以发送本页的安装提示词,或下载 Skill 包后解压到 ~/.claude/skills 或 ~/.codex/skills。

文本数据定量分析需要哪些权限?

该科研 Skill没有声明额外权限;执行具体操作时仍由 Agent 按自身规则请求确认。

文本数据定量分析经过官方审核吗?

是。SpeedSkills 收录的每个版本都会记录内容摘要与权限声明,当前版本已通过安全审核。