zhoujie/autoPaper
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Compare commits

base: zhoujie:0d74f0405a53c849f82af594cf22baa5ee75fbe9
Branches Tags
zhoujie:main
..
compare: zhoujie:c3e37f642a7665c8b7815ca7f213a79f251270f2
Branches Tags
zhoujie:main

No commits in common. "0d74f0405a53c849f82af594cf22baa5ee75fbe9" and "c3e37f642a7665c8b7815ca7f213a79f251270f2" have entirely different histories.
0d74f0405a ... c3e37f642a

19 changed files with 53 additions and 861 deletions
Show Stats Expand all files Collapse all files

82
.claude/skills/git-commit/SKILL.md
View File

@ -1,82 +0,0 @@
---
name: git-commit
description: Generate a Conventional-Commits-style message from the repo's current staged/unstaged changes and create the commit. Use whenever the user explicitly asks to commit, save, or checkpoint their work ("提交一下", "commit这些改动", "帮我提交代码", "存一下进度", "commit this"). Never invoke this proactively — only when the user asks for a commit, not merely after finishing an edit. Does not cover pushing, PR creation, amending, or any other git operation.
---
# Git 提交(规范化 commit message)
## 为什么需要这个技能
"提交一下"这句话背后其实有一套容易被跳过的步骤:先看清楚这次改动到底涉及
哪些文件、有没有不该进版本库的东西(密钥、临时文件、误生成的二进制),再把
分散的改动归纳成一句准确说明"为什么改"而不是"改了什么"的话——好的commit
message省的是未来翻`git log`时重新读一遍diff的时间。这个技能把这套步骤和
一套固定的message格式(Conventional Commits)定下来,避免每次现编、风格漂移。
## 工作流程
### 第一步:摸清这次改动的全貌
并行执行:
- `git status`(不要加`-uall`,大仓库会有性能问题)
- `git diff` 看未暂存的改动,`git diff --cached` 看已暂存的改动
- `git log --oneline -10` 看最近的提交,判断这个仓库习惯的message风格(有的仓库不用Conventional Commits前缀,这时候跟随仓库现有风格优先于本技能默认的格式)
### 第二步:提交前检查——发现问题就先说清楚,不要直接提交
逐条过一遍将要暂存的文件,确认没有:
- 明显的密钥/凭据文件(`.env`、`credentials.json`、私钥等)——发现了就跳过并提醒用户,不要连带提交
- 不该进版本库的产物:打包导出的二进制(`.zip`/`.skill`等)、编辑器/系统临时文件、体积异常大的文件
- 用`git add -A`/`git add .`扫进来的、用户可能还不想提交的无关改动——按文件名显式`git add`,不要笼统全加
发现可疑内容,先列出来问用户怎么处理,不要自作主张排除或包含。
### 第三步:生成 commit message
**message 正文(subject/body)用中文写**——除非第一步发现这个仓库历史上明显
是用英文写(且不是本技能自己产生的commit),这种情况下跟随仓库现有语言。
`type`前缀本身仍用英文标准词(`feat`/`fix`等,这是社区通用约定,不翻译),
`scope`用原始的模块/skill名(通常本身就是英文目录名,不需要翻译)。
格式采用 Conventional Commits:
```
<type>(<scope>): <中文subject>
<中文body — 可选,只在"为什么"不是一眼能看出来的时候写>
```
- `type` 从改动的实际性质里选,不要凭感觉:`feat`(新功能)、`fix`(修bug)、
`refactor`(不改行为的重构)、`docs`(纯文档)、`test`(测试)、`chore`(构建/
依赖/杂项)、`style`(纯格式,不改逻辑)——一次提交只涉及一种性质就只写一个
type;如果改动明显跨性质(比如"顺手"把无关重构也塞进了这次提交),提醒用户
考虑拆成多个提交,而不是硬凑一个笼统的type。
- `scope` 可选,涉及单一模块/skill时加上(比如`literature-search-verify`、
`paper-writing-grounded`),改动横跨全仓库时省略。
- `subject` 祈使/陈述语气均可、不超过~35个汉字、不以句号结尾,说清楚这次改动
做了什么,但真正的价值在于讲清楚**为什么**要这么改——如果"为什么"不是从
改动本身能一眼看出来的(比如修了一个不明显的bug、调整了一个非默认行为),
写进body里,不要只重复diff里已经能看到的内容。
- 不要在message里提及和这次改动无关的历史背景或猜测性动机。
### 第四步:暂存并提交
- 按第二步筛选后的文件列表显式`git add <file1> <file2> ...`,不要`-A`/`.`
- 用 heredoc 传message,保证多行格式正确:
```bash
git commit -m "$(cat <<'EOF'
<type>(<scope>): <subject>
<body>
EOF
)"
```
- 提交后跑一次`git status`确认工作区状态符合预期
### 第五步:红线
- 只在用户明确要求提交时才创建commit,不要在完成一次编辑后主动提交
- 不新建commit时优先于`--amend`——只有用户明确要求"修改上一个commit"才用amend,而且如果上一次是因为pre-commit hook失败而"没有真正提交成功",绝不能用`--amend`(那样会改到更早的一个commit)
- 不加`--no-verify`/`--no-gpg-sign`跳过hook或签名,除非用户明确要求;hook失败时先定位问题、修复后重新`git add`再建一个新commit
- 不主动`git push`,除非用户在这次请求里明确说了要推送
- 不做`reset --hard`/`checkout --`/`clean -f`等破坏性操作来"顺手"清理提交前发现的问题,发现了就报告给用户决定

60
.claude/skills/literature-search-verify/SKILL.md
View File

@ -1,6 +1,6 @@
--- ---
name: literature-search-verify name: literature-search-verify
description: Search academic literature across arXiv, Semantic Scholar, Crossref, OpenAlex, and other connected paper-search MCP tools, and independently verify every candidate citation before it is treated as real. Use this whenever the user asks to find papers, search literature on a topic, build a reading list, compile related-work references, check whether a citation or bibliography entry actually exists, or prepare references to import into Zotero or a .bib file — especially in academic writing contexts where a fabricated citation would be a real problem. Also covers guiding the user to the Zotero Connector browser extension for Chinese-language sources (CNKI/知网, Wanfang/万方, VIP/维普) that have no public API and cannot be reached by search tools. description: Search academic literature across arXiv, Semantic Scholar, Crossref, and other connected paper-search MCP tools, and independently verify every candidate citation before it is treated as real. Use this whenever the user asks to find papers, search literature on a topic, build a reading list, compile related-work references, check whether a citation or bibliography entry actually exists, or prepare references to import into Zotero or a .bib file — especially in academic writing contexts where a fabricated citation would be a real problem. Also covers guiding the user to the Zotero Connector browser extension for Chinese-language sources (CNKI/知网, Wanfang/万方, VIP/维普) that have no public API and cannot be reached by search tools.
--- ---
# 文献检索 + 反幻觉引用核查 # 文献检索 + 反幻觉引用核查
@ -25,34 +25,15 @@ description: Search academic literature across arXiv, Semantic Scholar, Crossref
`scripts/` 目录下已经写好了能直接跑的检索脚本,不依赖任何第三方Python包,也不需要装MCP工具: `scripts/` 目录下已经写好了能直接跑的检索脚本,不依赖任何第三方Python包,也不需要装MCP工具:
```bash ```bash
# 始终从项目根目录调用(脚本本身不关心cwd,但--bib-out等输出路径按项目根目录约定拼)。
# 一次性搞定:检索 arXiv + Semantic Scholar + Crossref,自动去重、逐条验证, # 一次性搞定:检索 arXiv + Semantic Scholar + Crossref,自动去重、逐条验证,
# 并把通过验证的条目写成BibTeX文件——这是应该默认调用的入口 # 并把通过验证的条目写成BibTeX文件——这是应该默认调用的入口
mkdir -p output/literature-search-verify/uav_aeromagnetic_compensation python3 scripts/literature_search.py "UAV magnetic compensation Tolles-Lawson" \
python3 .claude/skills/literature-search-verify/scripts/literature_search.py \ --max-per-source 8 --bib-out refs.bib
"UAV magnetic compensation Tolles-Lawson" \
--max-per-source 8 \
--bib-out output/literature-search-verify/uav_aeromagnetic_compensation/refs.bib
``` ```
草稿路径(检索报告JSON、下载的PDF、中间生成的bib)统一落在项目根目录下的 正常情况下**只需要跑这一条命令**,它内部会依次调用 `search_arxiv.py`、`search_semantic_scholar.py`、`search_crossref.py` 做检索,再对每条合并后的候选文献跑 `verify_citation.py` 做交叉验证,输出一份JSON报告(每条候选都带`verdict`字段)。如果只是想单独查一个来源,或者针对某一条文献单独复核,再分别调用对应的单个脚本(用法见每个脚本文件开头的docstring)。
`output/literature-search-verify/`,不要写回技能自己的目录里(`.claude/skills/`
应该只放技能代码,不放运行时产生的草稿数据)。如果项目里以后还有别的技能也
需要草稿区,各自建 `output/<skill-name>/` 子目录,互不混放。
**同一个技能内部也要按主题分开,不要所有检索会话都堆在同一层**:开始一个新 如果这些脚本因为网络原因跑不动(比如内网/代理限制导致连不上 arxiv.org、semanticscholar.org、crossref.org),`literature_search.py` 会把每个来源的报错单独记在`search_errors`里而不是直接崩溃——这时候老实告诉用户"检索脚本连不上网络,以下是报错信息",不要退回去凭记忆编文献。如果用户这边确实连不上这几个学术API域名,才退回到 web_search 工具,并在结果里明确标注"来自通用网络搜索的补充结果,未经过脚本的交叉验证流程,置信度较低"。
主题的检索时,先按"和第六步归档时同样的规则"把主题名转成 slug(小写、非
字母数字换成下划线,比如"UAV aeromagnetic compensation" → `uav_aeromagnetic_compensation`),
建 `output/literature-search-verify/<topic-slug>/` 子目录,这一整个主题下
不管跑多少轮检索、多少条不同的query,原始JSON/bib草稿都写进这一个子目录里
(文件名可以随意区分轮次,比如`q1.json`/`q2.json`),不要用不带主题区分的
通用文件名散落在`output/literature-search-verify/`根下。这样同一个主题的
草稿和第六步归档产出的`references/<topic-slug>/`目录能通过同一个slug对上号,
之后回来补充检索同一主题时也知道去哪个子目录续。
正常情况下**只需要跑这一条命令**,它内部会依次调用 `search_arxiv.py`、`search_semantic_scholar.py`、`search_crossref.py`、`search_openalex.py` 做检索,再对每条合并后的候选文献跑 `verify_citation.py` 做交叉验证,输出一份JSON报告(每条候选都带`verdict`字段)。如果只是想单独查一个来源,或者针对某一条文献单独复核,再分别调用对应的单个脚本(用法见每个脚本文件开头的docstring)。OpenAlex(`search_openalex.py`)是在 arXiv/Semantic Scholar/Crossref 之外新增的第四个源,免费、无需API key,覆盖面比单独的 Crossref 更广(聚合了包括 IEEE 在内的绝大多数出版商元数据),还能拿到开放获取PDF直链;可以设置`OPENALEX_MAILTO`环境变量为一个邮箱地址进入OpenAlex的"polite pool"换取更稳的限额,不设置也能正常用。
所有脚本的网络请求都经过 `http_utils.py` 统一封装:遇到 429(限流)或 5xx 会按指数退避自动重试几次(优先遵守服务端返回的`Retry-After`),不需要每次手动重试;`verify_citation.py`的跨源标题核查更是同时查 Semantic Scholar 和 OpenAlex 两个独立源、任一命中即算通过,不会再出现"S2一限流,整批候选的跨源核查全部退化成skipped"的情况(这是实际发生过的问题,现在已经用两个源+重试解决)。即便如此,如果这些脚本因为网络原因跑不动(比如内网/代理限制导致连不上这几个学术API域名),`literature_search.py` 会把每个来源的报错单独记在`search_errors`里而不是直接崩溃——这时候老实告诉用户"检索脚本连不上网络,以下是报错信息",不要退回去凭记忆编文献。如果用户这边确实连不上这几个学术API域名,才退回到 web_search 工具,并在结果里明确标注"来自通用网络搜索的补充结果,未经过脚本的交叉验证流程,置信度较低"。
如果用户已经连了 paper-search-mcp / scholar_mcp_server 这类MCP工具,可以补充用来扩大覆盖面(比如它们能覆盖PubMed、能直接下载PDF),但**不能替代**`verify_citation.py`的交叉验证这一步——MCP搜到的候选一样要过一遍验证,不能因为是工具搜出来的就默认可信。 如果用户已经连了 paper-search-mcp / scholar_mcp_server 这类MCP工具,可以补充用来扩大覆盖面(比如它们能覆盖PubMed、能直接下载PDF),但**不能替代**`verify_citation.py`的交叉验证这一步——MCP搜到的候选一样要过一遍验证,不能因为是工具搜出来的就默认可信。
@ -62,7 +43,7 @@ python3 .claude/skills/literature-search-verify/scripts/literature_search.py \
1. **arXiv ID 独立核实**:如果有 arXiv ID,反查一次 arXiv API,确认这个ID真的存在且标题对得上——一个编造的ID在这一步会直接暴露。 1. **arXiv ID 独立核实**:如果有 arXiv ID,反查一次 arXiv API,确认这个ID真的存在且标题对得上——一个编造的ID在这一步会直接暴露。
2. **DOI 独立核实**:如果有 DOI,反查一次 Crossref,确认这个 DOI 真的能解析出对应文献。 2. **DOI 独立核实**:如果有 DOI,反查一次 Crossref,确认这个 DOI 真的能解析出对应文献。
3. **跨源标题复核(双源)**:不管有没有ID,单独拿标题分别去 Semantic Scholar 和 OpenAlex 各搜一次,要求返回的标题跟候选标题高度相似(相似度≥0.9)——这一步专门用来抓"标题作者读起来很像真的,但其实是编出来的"这种情况。两个源里任一个命中相似度达标就算通过;只有当两个源都明确没找到匹配(不是因为网络问题被跳过)时才算这一项核查失败——避免某个源覆盖不全(比如论文太新还没被其中一个索引收录)被误判成"编造"。 3. **跨源标题复核**:不管有没有ID,单独拿标题去 Semantic Scholar 搜一次,要求返回的标题跟候选标题高度相似(相似度≥0.9)——这一步专门用来抓"标题作者读起来很像真的,但其实是编出来的"这种情况。
每条候选最后会带一个`verdict`: 每条候选最后会带一个`verdict`:
- **verified**:至少一项独立核查通过,而且没有任何一项核查明确失败 - **verified**:至少一项独立核查通过,而且没有任何一项核查明确失败
@ -85,19 +66,15 @@ MCP 检索工具覆盖的是 arXiv/Semantic Scholar/Crossref 这类有公开 API
### 第六步:归档 ### 第六步:归档
`<project-root>/output/literature-search-verify/` 只是脚本运行时的草稿区——里面 `output/` 目录只是脚本运行时的草稿区——里面混着每一轮探索性检索的原始JSON(包括被过滤掉的噪声,比如"Tolles""Lawson"被当成人名匹配出的无关文献),不适合作为最终交付物,而且随着会话增多会越堆越乱、也不方便下次会话或用户直接翻阅。
混着每一轮探索性检索的原始JSON(包括被过滤掉的噪声,比如"Tolles""Lawson"被当成
人名匹配出的无关文献),不适合作为最终交付物,而且随着会话增多会越堆越乱、也不
方便下次会话或用户直接翻阅。
所以每次整理出一份**稳定可信的参考文献列表**(不管是第一轮检索还是后续多轮补充检索合并后的结果)之后,调用归档脚本把它固化到项目级目录,而不是留在草稿区里: 所以每次整理出一份**稳定可信的参考文献列表**(不管是第一轮检索还是后续多轮补充检索合并后的结果)之后,调用归档脚本把它固化到项目级目录,而不是留在技能自己的`output/`里:
```bash ```bash
python3 .claude/skills/literature-search-verify/scripts/archive_references.py \ python3 scripts/archive_references.py "UAV aeromagnetic compensation" \
"UAV aeromagnetic compensation" \ --bib output/uav_aeromagnetic_compensation_final.bib \
--bib output/literature-search-verify/uav_aeromagnetic_compensation/uav_aeromagnetic_compensation_final.bib \
--project-root . \ --project-root . \
--pdfs-dir output/literature-search-verify/uav_aeromagnetic_compensation/pdfs \ --pdfs-dir output/pdfs \
--suspect "某条可疑文献标题|不建议引用的具体原因" \ --suspect "某条可疑文献标题|不建议引用的具体原因" \
--notes "检索覆盖了哪些方向、哪些方向搜了但没结果、中文文献缺口提醒等" --notes "检索覆盖了哪些方向、哪些方向搜了但没结果、中文文献缺口提醒等"
``` ```
@ -112,21 +89,6 @@ python3 .claude/skills/literature-search-verify/scripts/archive_references.py \
- 同一个`topic`名字多次调用会往同一个归档目录里覆盖更新(bib和README会被覆盖,pdfs按文件名去重合并),所以后续检索到更多文献后可以直接对同一个topic重新跑一遍归档脚本来更新,不需要手动合并。 - 同一个`topic`名字多次调用会往同一个归档目录里覆盖更新(bib和README会被覆盖,pdfs按文件名去重合并),所以后续检索到更多文献后可以直接对同一个topic重新跑一遍归档脚本来更新,不需要手动合并。
- 这一步做完之后可以明确告诉用户归档目录的路径,方便他们后续在`paper-writing-grounded`阶段直接引用。 - 这一步做完之后可以明确告诉用户归档目录的路径,方便他们后续在`paper-writing-grounded`阶段直接引用。
## 维护者备注:单元测试
`scripts/tests/` 下有针对核心正确性逻辑的单元测试(不发真实网络请求,全部用
mock):`verify_citation.py` 的三档判定和双源标题核查合并逻辑、
`archive_references.py` 的 bib 解析(含嵌套花括号、中文主题名slugify等边界
情况)、`literature_search.py` 的候选去重合并与BibTeX生成、`http_utils.py`
的限流重试。改动这几个脚本后应该跑一遍:
```bash
python3 -m unittest discover -s .claude/skills/literature-search-verify/scripts/tests -t .claude/skills/literature-search-verify/scripts
```
只用标准库`unittest`,不引入pytest等第三方测试框架,和脚本本身"不依赖第三方
包"的原则保持一致。
## 和 paper-writing-grounded 技能的配合 ## 和 paper-writing-grounded 技能的配合
这个技能负责把"真实存在、经过核实的文献"整理好并生成 BibTeX;写作阶段的 paper-writing-grounded 技能会直接消费这里产出的 citation key,正文引用只能来自这里核实过的条目,不会凭空生成新的引用。两个技能配合使用时,建议先跑完这个技能、拿到稳定的参考文献列表,再进入写作。 这个技能负责把"真实存在、经过核实的文献"整理好并生成 BibTeX;写作阶段的 paper-writing-grounded 技能会直接消费这里产出的 citation key,正文引用只能来自这里核实过的条目,不会凭空生成新的引用。两个技能配合使用时,建议先跑完这个技能、拿到稳定的参考文献列表,再进入写作。

17
.claude/skills/literature-search-verify/scripts/archive_references.py
View File

@ -13,14 +13,11 @@ what's there without re-reading the conversation that produced it.
No third-party dependencies; uses only the standard library. No third-party dependencies; uses only the standard library.
CLI usage (run from the project root; scratch input lives under CLI usage:
output/literature-search-verify/<topic-slug>/, matching the slug this python3 archive_references.py "UAV aeromagnetic compensation" \\
script derives from the topic argument): --bib output/uav_aeromagnetic_compensation_final.bib \\
python3 .claude/skills/literature-search-verify/scripts/archive_references.py \\
"UAV aeromagnetic compensation" \\
--bib output/literature-search-verify/uav_aeromagnetic_compensation/uav_aeromagnetic_compensation_final.bib \\
--project-root . \\ --project-root . \\
--pdfs-dir output/literature-search-verify/uav_aeromagnetic_compensation/pdfs \\ --pdfs-dir output/pdfs \\
--suspect "Some fabricated-looking title|DOI resolves but venue is topically unrelated" \\ --suspect "Some fabricated-looking title|DOI resolves but venue is topically unrelated" \\
--notes "Kalman-filter and GA/PSO angles searched, no on-topic hits found." --notes "Kalman-filter and GA/PSO angles searched, no on-topic hits found."
@ -35,12 +32,8 @@ from datetime import date
def slugify(text): def slugify(text):
"""ASCII gets lowercased; CJK characters are kept as-is (there's no
reversible case-fold for them) so an all-Chinese topic name doesn't
collapse to nothing and fall back to the generic "references" name,
colliding with every other Chinese-named topic archived in this repo."""
text = text.strip().lower() text = text.strip().lower()
text = re.sub(r"[^a-z0-9一-鿿]+", "_", text) text = re.sub(r"[^a-z0-9]+", "_", text)
return text.strip("_")[:60] or "references" return text.strip("_")[:60] or "references"

46
.claude/skills/literature-search-verify/scripts/http_utils.py
View File

@ -1,46 +0,0 @@
#!/usr/bin/env python3
"""
Shared HTTP GET helper with exponential-backoff retry for rate limiting (429)
and transient server errors (5xx). Stdlib only.
Every search_*.py / verify_citation.py network call should go through this
instead of calling urllib.request.urlopen directly, so a single source being
temporarily rate-limited doesn't quietly degrade a whole batch of candidates
to "unverified" the way Semantic Scholar's 429s did before this existed.
Importable:
from http_utils import fetch
"""
import time
import urllib.request
import urllib.error
RETRYABLE_CODES = {429, 500, 502, 503, 504}
def _wait_seconds(http_error, attempt, base=1.0, cap=20.0):
retry_after = http_error.headers.get("Retry-After") if http_error.headers else None
if retry_after:
try:
return min(float(retry_after), cap)
except ValueError:
pass
return min(base * (2 ** attempt), cap)
def fetch(url, headers=None, timeout=20, max_retries=4):
"""GET url, retrying on 429/5xx with exponential backoff (honoring a
Retry-After header when the server sends one). Returns the raw response
bytes. Raises the underlying urllib error if retries are exhausted or the
error isn't retryable (e.g. 404)."""
req = urllib.request.Request(url, headers=headers or {})
attempt = 0
while True:
try:
with urllib.request.urlopen(req, timeout=timeout) as resp:
return resp.read()
except urllib.error.HTTPError as e:
if e.code not in RETRYABLE_CODES or attempt >= max_retries:
raise
time.sleep(_wait_seconds(e, attempt))
attempt += 1

16
.claude/skills/literature-search-verify/scripts/literature_search.py
View File

@ -1,9 +1,8 @@
#!/usr/bin/env python3 #!/usr/bin/env python3
""" """
End-to-end literature search: query arXiv + Semantic Scholar + Crossref + End-to-end literature search: query arXiv + Semantic Scholar + Crossref,
OpenAlex, merge/dedupe candidates, independently verify each one, and emit merge/dedupe candidates, independently verify each one, and emit both a
both a human-readable report and BibTeX for the entries that passed human-readable report and BibTeX for the entries that passed verification.
verification.
This is the one script Claude should actually call for a normal literature This is the one script Claude should actually call for a normal literature
search -- the individual search_*.py / verify_citation.py scripts exist search -- the individual search_*.py / verify_citation.py scripts exist
@ -29,7 +28,6 @@ sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
from search_arxiv import search_arxiv from search_arxiv import search_arxiv
from search_semantic_scholar import search_s2 from search_semantic_scholar import search_s2
from search_crossref import search_crossref from search_crossref import search_crossref
from search_openalex import search_openalex
from verify_citation import verify from verify_citation import verify
@ -119,13 +117,7 @@ def to_bibtex(candidate, key):
def run(query, max_per_source=8): def run(query, max_per_source=8):
all_results = [] all_results = []
errors = {} errors = {}
sources = ( for name, fn in (("arxiv", search_arxiv), ("semantic_scholar", search_s2), ("crossref", search_crossref)):
("arxiv", search_arxiv),
("semantic_scholar", search_s2),
("crossref", search_crossref),
("openalex", search_openalex),
)
for name, fn in sources:
try: try:
all_results.extend(fn(query, max_per_source)) all_results.extend(fn(query, max_per_source))
except Exception as e: except Exception as e:

8
.claude/skills/literature-search-verify/scripts/search_arxiv.py
View File

@ -9,15 +9,12 @@ Importable:
from search_arxiv import search_arxiv from search_arxiv import search_arxiv
""" """
import sys import sys
import os
import json import json
import argparse import argparse
import urllib.request
import urllib.parse import urllib.parse
import xml.etree.ElementTree as ET import xml.etree.ElementTree as ET
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
from http_utils import fetch
ARXIV_API = "http://export.arxiv.org/api/query" ARXIV_API = "http://export.arxiv.org/api/query"
NS = {"atom": "http://www.w3.org/2005/Atom"} NS = {"atom": "http://www.w3.org/2005/Atom"}
@ -31,7 +28,8 @@ def search_arxiv(query, max_results=10, timeout=20):
"sortOrder": "descending", "sortOrder": "descending",
} }
url = f"{ARXIV_API}?{urllib.parse.urlencode(params)}" url = f"{ARXIV_API}?{urllib.parse.urlencode(params)}"
data = fetch(url, timeout=timeout) with urllib.request.urlopen(url, timeout=timeout) as resp:
data = resp.read()
root = ET.fromstring(data) root = ET.fromstring(data)
results = [] results = []
for entry in root.findall("atom:entry", NS): for entry in root.findall("atom:entry", NS):

9
.claude/skills/literature-search-verify/scripts/search_crossref.py
View File

@ -10,14 +10,11 @@ Importable:
from search_crossref import search_crossref from search_crossref import search_crossref
""" """
import sys import sys
import os
import json import json
import argparse import argparse
import urllib.request
import urllib.parse import urllib.parse
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
from http_utils import fetch
CROSSREF_API = "https://api.crossref.org/works" CROSSREF_API = "https://api.crossref.org/works"
UA = "literature-search-verify-skill/1.0 (mailto:research-assistant@example.com)" UA = "literature-search-verify-skill/1.0 (mailto:research-assistant@example.com)"
@ -25,7 +22,9 @@ UA = "literature-search-verify-skill/1.0 (mailto:research-assistant@example.com)
def search_crossref(query, max_results=10, timeout=20): def search_crossref(query, max_results=10, timeout=20):
params = {"query": query, "rows": max_results} params = {"query": query, "rows": max_results}
url = f"{CROSSREF_API}?{urllib.parse.urlencode(params)}" url = f"{CROSSREF_API}?{urllib.parse.urlencode(params)}"
data = json.loads(fetch(url, headers={"User-Agent": UA}, timeout=timeout)) req = urllib.request.Request(url, headers={"User-Agent": UA})
with urllib.request.urlopen(req, timeout=timeout) as resp:
data = json.loads(resp.read())
results = [] results = []
for item in data.get("message", {}).get("items", []) or []: for item in data.get("message", {}).get("items", []) or []:
titles = item.get("title") or [] titles = item.get("title") or []

92
.claude/skills/literature-search-verify/scripts/search_openalex.py
View File

@ -1,92 +0,0 @@
#!/usr/bin/env python3
"""
Search the OpenAlex Works API. No API key required -- set the OPENALEX_MAILTO
environment variable to an email address to use OpenAlex's "polite pool" for
higher and more reliable rate limits (OpenAlex's own recommendation, not a
credential). Coverage is broader than Crossref alone (includes arXiv, IEEE,
and most publishers' metadata in one place) and each result carries an
open-access PDF link when one exists.
CLI usage:
python3 search_openalex.py "UAV magnetic compensation" --max 10
Importable:
from search_openalex import search_openalex
"""
import sys
import os
import json
import argparse
import urllib.parse
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
from http_utils import fetch
OPENALEX_API = "https://api.openalex.org/works"
SELECT_FIELDS = ",".join([
"doi", "title", "display_name", "publication_year", "authorships",
"primary_location", "open_access", "cited_by_count", "abstract_inverted_index",
])
def _reconstruct_abstract(inverted_index):
"""OpenAlex returns abstracts as a word -> [positions] inverted index
(for copyright reasons) instead of plain text; rebuild the plain text."""
if not inverted_index:
return None
positions = {}
for word, idxs in inverted_index.items():
for i in idxs:
positions[i] = word
if not positions:
return None
return " ".join(positions[i] for i in sorted(positions))
def search_openalex(query, max_results=10, timeout=20):
params = {"search": query, "per_page": max_results, "select": SELECT_FIELDS}
mailto = os.environ.get("OPENALEX_MAILTO")
if mailto:
params["mailto"] = mailto
url = f"{OPENALEX_API}?{urllib.parse.urlencode(params)}"
data = json.loads(fetch(url, timeout=timeout))
results = []
for item in data.get("results", []) or []:
title = item.get("title") or item.get("display_name")
authors = [
a.get("author", {}).get("display_name")
for a in (item.get("authorships") or [])
if a.get("author")
]
doi = item.get("doi")
if doi and doi.startswith("https://doi.org/"):
doi = doi[len("https://doi.org/"):]
primary_location = item.get("primary_location") or {}
source = primary_location.get("source") or {}
open_access = item.get("open_access") or {}
results.append({
"source": "openalex",
"title": title,
"authors": authors,
"year": item.get("publication_year"),
"venue": source.get("display_name"),
"doi": doi,
"arxiv_id": None,
"abstract": _reconstruct_abstract(item.get("abstract_inverted_index")),
"citation_count": item.get("cited_by_count"),
"pdf_url": open_access.get("oa_url") or primary_location.get("pdf_url"),
})
return results
if __name__ == "__main__":
ap = argparse.ArgumentParser(description=__doc__)
ap.add_argument("query")
ap.add_argument("--max", type=int, default=10)
args = ap.parse_args()
try:
out = search_openalex(args.query, args.max)
print(json.dumps(out, ensure_ascii=False, indent=2))
except Exception as e:
print(json.dumps({"error": str(e)}, ensure_ascii=False))
sys.exit(1)

11
.claude/skills/literature-search-verify/scripts/search_semantic_scholar.py
View File

@ -13,11 +13,9 @@ import sys
import os import os
import json import json
import argparse import argparse
import urllib.request
import urllib.parse import urllib.parse
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
from http_utils import fetch
S2_API = "https://api.semanticscholar.org/graph/v1/paper/search" S2_API = "https://api.semanticscholar.org/graph/v1/paper/search"
FIELDS = "title,authors,year,venue,externalIds,abstract,citationCount" FIELDS = "title,authors,year,venue,externalIds,abstract,citationCount"
@ -25,11 +23,12 @@ FIELDS = "title,authors,year,venue,externalIds,abstract,citationCount"
def search_s2(query, max_results=10, timeout=20): def search_s2(query, max_results=10, timeout=20):
params = {"query": query, "limit": max_results, "fields": FIELDS} params = {"query": query, "limit": max_results, "fields": FIELDS}
url = f"{S2_API}?{urllib.parse.urlencode(params)}" url = f"{S2_API}?{urllib.parse.urlencode(params)}"
headers = {} req = urllib.request.Request(url)
api_key = os.environ.get("S2_API_KEY") api_key = os.environ.get("S2_API_KEY")
if api_key: if api_key:
headers["x-api-key"] = api_key req.add_header("x-api-key", api_key)
data = json.loads(fetch(url, headers=headers, timeout=timeout)) with urllib.request.urlopen(req, timeout=timeout) as resp:
data = json.loads(resp.read())
results = [] results = []
for p in data.get("data", []) or []: for p in data.get("data", []) or []:
ext = p.get("externalIds") or {} ext = p.get("externalIds") or {}

0
.claude/skills/literature-search-verify/scripts/tests/__init__.py
View File

99
.claude/skills/literature-search-verify/scripts/tests/test_archive_references.py
View File

@ -1,99 +0,0 @@
#!/usr/bin/env python3
"""
Unit tests for archive_references.py's slugify() and the minimal bib parser,
covering the edge cases that are easy to get wrong silently: CJK topic names
and nested-brace LaTeX field values (e.g. \\ensuremath{\\epsilon}).
Run: python -m unittest discover -s .claude/skills/literature-search-verify/scripts
"""
import os
import sys
import tempfile
import unittest
sys.path.insert(0, os.path.join(os.path.dirname(os.path.abspath(__file__)), ".."))
import archive_references as ar
class TestSlugify(unittest.TestCase):
def test_ascii_topic(self):
self.assertEqual(ar.slugify("UAV aeromagnetic compensation"), "uav_aeromagnetic_compensation")
def test_chinese_topic_is_preserved_not_collapsed_to_generic_name(self):
slug = ar.slugify("无人机磁补偿")
self.assertNotEqual(slug, "references")
self.assertIn("补偿", slug)
def test_mixed_chinese_and_english(self):
slug = ar.slugify("无人机磁补偿 Tolles-Lawson")
self.assertTrue(slug.endswith("tolles_lawson"))
def test_blank_topic_falls_back_to_references(self):
self.assertEqual(ar.slugify(" "), "references")
def test_truncated_to_60_chars(self):
self.assertEqual(len(ar.slugify("a" * 100)), 60)
class TestParseBibEntries(unittest.TestCase):
def _write_bib(self, content):
fd, path = tempfile.mkstemp(suffix=".bib")
with os.fdopen(fd, "w", encoding="utf-8") as f:
f.write(content)
self.addCleanup(os.remove, path)
return path
def test_nested_braces_in_field_value_do_not_truncate_it(self):
bib = (
"@article{wu2017,\n"
" title = {Aeromagnetic gradient compensation using \\ensuremath{\\epsilon}-SVR},\n"
" journal = {Journal of Applied Remote Sensing},\n"
" year = {2017},\n"
" doi = {10.1117/1.jrs.11.025012},\n"
"}"
)
entries = ar.parse_bib_entries(self._write_bib(bib))
self.assertEqual(len(entries), 1)
self.assertIn("\\ensuremath{\\epsilon}-SVR", entries[0]["title"])
self.assertEqual(entries[0]["year"], "2017")
def test_multiple_entries_and_optional_fields(self):
bib = (
"@article{a2020,\n"
" title = {First paper},\n"
" year = {2020},\n"
"}\n\n"
"@article{b2021,\n"
" title = {Second paper},\n"
" year = {2021},\n"
" note = {some note},\n"
"}"
)
entries = ar.parse_bib_entries(self._write_bib(bib))
self.assertEqual([e["key"] for e in entries], ["a2020", "b2021"])
self.assertNotIn("note", entries[0])
self.assertEqual(entries[1]["note"], "some note")
class TestYearSortKey(unittest.TestCase):
def test_entries_without_a_parseable_year_sort_last(self):
entries = [{"key": "b", "year": ""}, {"key": "a", "year": "1999"}]
entries.sort(key=ar.year_sort_key)
self.assertEqual([e["key"] for e in entries], ["a", "b"])
class TestBuildReadme(unittest.TestCase):
def test_omits_optional_sections_when_not_given(self):
readme = ar.build_readme("Topic", [], 0, [], None)
self.assertNotIn("Flagged during search", readme)
self.assertNotIn("Search coverage notes", readme)
def test_includes_suspect_and_notes_when_given(self):
readme = ar.build_readme("Topic", [], 0, ["Bad title|dubious venue"], "coverage notes here")
self.assertIn("Bad title", readme)
self.assertIn("dubious venue", readme)
self.assertIn("coverage notes here", readme)
if __name__ == "__main__":
unittest.main()

103
.claude/skills/literature-search-verify/scripts/tests/test_http_utils.py
View File

@ -1,103 +0,0 @@
#!/usr/bin/env python3
"""
Unit tests for http_utils.fetch's retry/backoff behavior. All network calls
are mocked -- these never touch the real network.
Run: python -m unittest discover -s .claude/skills/literature-search-verify/scripts
"""
import os
import sys
import unittest
import urllib.error
import urllib.request
from email.message import Message
from unittest import mock
sys.path.insert(0, os.path.join(os.path.dirname(os.path.abspath(__file__)), ".."))
import http_utils
def _http_error(code, retry_after=None):
hdrs = Message()
if retry_after is not None:
hdrs["Retry-After"] = str(retry_after)
return urllib.error.HTTPError("http://example.com", code, "err", hdrs, None)
class FakeResponse:
def __init__(self, data):
self._data = data
def read(self):
return self._data
def __enter__(self):
return self
def __exit__(self, *exc_info):
return False
class TestFetch(unittest.TestCase):
@mock.patch("urllib.request.urlopen")
@mock.patch("http_utils.time.sleep", return_value=None)
def test_succeeds_first_try_without_sleeping(self, mock_sleep, mock_urlopen):
mock_urlopen.return_value = FakeResponse(b"ok")
result = http_utils.fetch("http://example.com")
self.assertEqual(result, b"ok")
mock_sleep.assert_not_called()
@mock.patch("urllib.request.urlopen")
@mock.patch("http_utils.time.sleep", return_value=None)
def test_retries_on_429_then_succeeds(self, mock_sleep, mock_urlopen):
mock_urlopen.side_effect = [_http_error(429), FakeResponse(b"ok")]
result = http_utils.fetch("http://example.com", max_retries=3)
self.assertEqual(result, b"ok")
self.assertEqual(mock_urlopen.call_count, 2)
mock_sleep.assert_called_once()
@mock.patch("urllib.request.urlopen")
@mock.patch("http_utils.time.sleep", return_value=None)
def test_retries_on_5xx(self, mock_sleep, mock_urlopen):
mock_urlopen.side_effect = [_http_error(503), FakeResponse(b"ok")]
result = http_utils.fetch("http://example.com", max_retries=3)
self.assertEqual(result, b"ok")
@mock.patch("urllib.request.urlopen")
@mock.patch("http_utils.time.sleep", return_value=None)
def test_raises_after_exhausting_retries(self, mock_sleep, mock_urlopen):
mock_urlopen.side_effect = [_http_error(429)] * 10
with self.assertRaises(urllib.error.HTTPError):
http_utils.fetch("http://example.com", max_retries=2)
self.assertEqual(mock_urlopen.call_count, 3) # initial attempt + 2 retries
@mock.patch("urllib.request.urlopen")
@mock.patch("http_utils.time.sleep", return_value=None)
def test_non_retryable_status_raises_immediately(self, mock_sleep, mock_urlopen):
mock_urlopen.side_effect = _http_error(404)
with self.assertRaises(urllib.error.HTTPError):
http_utils.fetch("http://example.com")
self.assertEqual(mock_urlopen.call_count, 1)
mock_sleep.assert_not_called()
class TestWaitSeconds(unittest.TestCase):
def test_honors_retry_after_header(self):
err = _http_error(429, retry_after=5)
self.assertEqual(http_utils._wait_seconds(err, attempt=0), 5.0)
def test_falls_back_to_exponential_backoff_without_header(self):
err = _http_error(429)
self.assertAlmostEqual(http_utils._wait_seconds(err, attempt=2, base=1.0), 4.0)
def test_caps_wait_time(self):
err = _http_error(429, retry_after=999)
self.assertEqual(http_utils._wait_seconds(err, attempt=0, cap=20.0), 20.0)
def test_ignores_unparseable_retry_after(self):
err = _http_error(429, retry_after="not-a-number")
self.assertAlmostEqual(http_utils._wait_seconds(err, attempt=0, base=1.0), 1.0)
if __name__ == "__main__":
unittest.main()

67
.claude/skills/literature-search-verify/scripts/tests/test_literature_search.py
View File

@ -1,67 +0,0 @@
#!/usr/bin/env python3
"""
Unit tests for literature_search.py's candidate merging and BibTeX generation.
Run: python -m unittest discover -s .claude/skills/literature-search-verify/scripts
"""
import os
import sys
import unittest
sys.path.insert(0, os.path.join(os.path.dirname(os.path.abspath(__file__)), ".."))
import literature_search as ls
class TestMergeCandidates(unittest.TestCase):
def test_dedupes_same_paper_across_sources_and_fills_missing_fields(self):
items = [
{"source": "arxiv", "title": "Deep Learning for Aeromagnetic Compensation",
"arxiv_id": "2401.00001", "doi": None, "year": "2024"},
{"source": "crossref", "title": "Deep Learning for Aeromagnetic Compensation",
"arxiv_id": None, "doi": "10.1/x", "year": 2024, "venue": "Some Journal"},
]
merged = ls.merge_candidates(items)
self.assertEqual(len(merged), 1)
self.assertEqual(merged[0]["arxiv_id"], "2401.00001")
self.assertEqual(merged[0]["doi"], "10.1/x")
self.assertEqual(sorted(merged[0]["sources"]), ["arxiv", "crossref"])
def test_distinct_titles_are_not_merged(self):
items = [
{"source": "arxiv", "title": "Paper About Cats"},
{"source": "crossref", "title": "Paper About Dogs"},
]
self.assertEqual(len(ls.merge_candidates(items)), 2)
class TestBibtexKey(unittest.TestCase):
def test_collision_gets_letter_suffix(self):
used = set()
key1 = ls.make_bibtex_key({"authors": ["Jane Smith"], "year": "2020"}, used)
key2 = ls.make_bibtex_key({"authors": ["John Smith"], "year": "2020"}, used)
self.assertEqual(key1, "smith2020")
self.assertEqual(key2, "smith2020a")
def test_no_authors_falls_back_to_unknown(self):
key = ls.make_bibtex_key({"authors": [], "year": "2020"}, set())
self.assertEqual(key, "unknown2020")
class TestToBibtex(unittest.TestCase):
def test_arxiv_only_candidate_uses_misc_with_eprint(self):
cand = {"authors": ["A B"], "title": "T", "year": "2024",
"arxiv_id": "2401.00001", "venue": None, "doi": None}
bib = ls.to_bibtex(cand, "b2024")
self.assertTrue(bib.startswith("@misc{b2024,"))
self.assertIn("eprint = {2401.00001}", bib)
def test_candidate_with_venue_uses_article_with_doi(self):
cand = {"authors": ["A B"], "title": "T", "year": "2024",
"arxiv_id": None, "venue": "Some Journal", "doi": "10.1/x"}
bib = ls.to_bibtex(cand, "b2024")
self.assertTrue(bib.startswith("@article{b2024,"))
self.assertIn("doi = {10.1/x}", bib)
if __name__ == "__main__":
unittest.main()

94
.claude/skills/literature-search-verify/scripts/tests/test_verify_citation.py
View File

@ -1,94 +0,0 @@
#!/usr/bin/env python3
"""
Unit tests for verify_citation's verdict aggregation logic -- this is the
anti-hallucination guarantee, so its three-way verified/suspect/unverified
decision needs to stay correct independent of any real network call. All
individual check_* functions are mocked; only the aggregation logic itself
(verify() and _combine_title_checks()) is under test.
Run: python -m unittest discover -s .claude/skills/literature-search-verify/scripts
"""
import os
import sys
import unittest
from unittest import mock
sys.path.insert(0, os.path.join(os.path.dirname(os.path.abspath(__file__)), ".."))
import verify_citation as vc
class TestCombineTitleChecks(unittest.TestCase):
"""Two independent title sources (S2 + OpenAlex) shouldn't let one
source's incomplete coverage outvote the other's confirmation."""
def test_both_pass_picks_higher_similarity(self):
s2 = {"status": "pass", "similarity": 0.95, "matched_title": "A"}
oa = {"status": "pass", "similarity": 0.99, "matched_title": "B"}
combined = vc._combine_title_checks(s2, oa)
self.assertEqual(combined["status"], "pass")
self.assertEqual(combined["matched_title"], "B")
def test_one_pass_one_fail_still_passes(self):
s2 = {"status": "fail", "reason": "no matching title found on Semantic Scholar"}
oa = {"status": "pass", "similarity": 0.95, "matched_title": "B"}
self.assertEqual(vc._combine_title_checks(s2, oa)["status"], "pass")
def test_one_pass_one_skipped_still_passes(self):
s2 = {"status": "skipped", "reason": "timeout"}
oa = {"status": "pass", "similarity": 0.95, "matched_title": "B"}
self.assertEqual(vc._combine_title_checks(s2, oa)["status"], "pass")
def test_both_fail_is_fail(self):
s2 = {"status": "fail", "reason": "x"}
oa = {"status": "fail", "reason": "y"}
self.assertEqual(vc._combine_title_checks(s2, oa)["status"], "fail")
def test_fail_plus_skipped_is_still_fail(self):
s2 = {"status": "fail", "reason": "x"}
oa = {"status": "skipped", "reason": "y"}
self.assertEqual(vc._combine_title_checks(s2, oa)["status"], "fail")
def test_both_skipped_is_skipped_not_fail(self):
s2 = {"status": "skipped", "reason": "x"}
oa = {"status": "skipped", "reason": "y"}
self.assertEqual(vc._combine_title_checks(s2, oa)["status"], "skipped")
class TestVerifyVerdict(unittest.TestCase):
def test_all_checks_pass_is_verified(self):
with mock.patch.object(vc, "check_arxiv_id", return_value={"status": "pass"}), \
mock.patch.object(vc, "check_doi", return_value={"status": "pass"}), \
mock.patch.object(vc, "check_title_cross_source_s2", return_value={"status": "pass", "similarity": 1.0}), \
mock.patch.object(vc, "check_title_cross_source_openalex", return_value={"status": "pass", "similarity": 1.0}):
result = vc.verify(title="T", arxiv_id="1234.5678", doi="10.1/x")
self.assertEqual(result["verdict"], "verified")
def test_one_failing_check_is_suspect_even_if_another_passes(self):
with mock.patch.object(vc, "check_arxiv_id", return_value={"status": "fail", "reason": "arXiv ID not found"}), \
mock.patch.object(vc, "check_doi", return_value={"status": "pass"}):
result = vc.verify(arxiv_id="9999.99999", doi="10.1/x")
self.assertEqual(result["verdict"], "suspect")
def test_all_checks_skipped_is_unverified_not_verified(self):
with mock.patch.object(vc, "check_arxiv_id", return_value={"status": "skipped", "reason": "timeout"}):
result = vc.verify(arxiv_id="1234.5678")
self.assertEqual(result["verdict"], "unverified")
def test_no_identifiers_given_produces_no_checks_and_unverified(self):
result = vc.verify(title=None, arxiv_id=None, doi=None)
self.assertEqual(result["checks"], {})
self.assertEqual(result["verdict"], "unverified")
class TestTitleSimilarity(unittest.TestCase):
def test_identical_titles_score_one(self):
self.assertEqual(vc.title_similarity("Same Title", "same title"), 1.0)
def test_empty_or_missing_inputs_score_zero(self):
self.assertEqual(vc.title_similarity("", "x"), 0.0)
self.assertEqual(vc.title_similarity(None, "x"), 0.0)
self.assertEqual(vc.title_similarity("x", None), 0.0)
if __name__ == "__main__":
unittest.main()

84
.claude/skills/literature-search-verify/scripts/verify_citation.py
View File

@ -13,17 +13,14 @@ Importable:
from verify_citation import verify from verify_citation import verify
""" """
import sys import sys
import os
import json import json
import argparse import argparse
import difflib import difflib
import urllib.request
import urllib.parse import urllib.parse
import urllib.error import urllib.error
import xml.etree.ElementTree as ET import xml.etree.ElementTree as ET
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
from http_utils import fetch
ATOM_NS = {"atom": "http://www.w3.org/2005/Atom"} ATOM_NS = {"atom": "http://www.w3.org/2005/Atom"}
@ -37,7 +34,8 @@ def check_arxiv_id(arxiv_id, timeout=20):
"""Confirm an arXiv ID actually resolves to a real paper.""" """Confirm an arXiv ID actually resolves to a real paper."""
try: try:
url = f"http://export.arxiv.org/api/query?id_list={urllib.parse.quote(arxiv_id)}" url = f"http://export.arxiv.org/api/query?id_list={urllib.parse.quote(arxiv_id)}"
data = fetch(url, timeout=timeout) with urllib.request.urlopen(url, timeout=timeout) as resp:
data = resp.read()
root = ET.fromstring(data) root = ET.fromstring(data)
entry = root.find("atom:entry", ATOM_NS) entry = root.find("atom:entry", ATOM_NS)
if entry is None: if entry is None:
@ -53,7 +51,11 @@ def check_doi(doi, timeout=20):
"""Confirm a DOI actually resolves via Crossref.""" """Confirm a DOI actually resolves via Crossref."""
try: try:
url = f"https://api.crossref.org/works/{urllib.parse.quote(doi)}" url = f"https://api.crossref.org/works/{urllib.parse.quote(doi)}"
data = json.loads(fetch(url, headers={"User-Agent": "literature-search-verify-skill/1.0"}, timeout=timeout)) req = urllib.request.Request(
url, headers={"User-Agent": "literature-search-verify-skill/1.0"}
)
with urllib.request.urlopen(req, timeout=timeout) as resp:
data = json.loads(resp.read())
titles = data.get("message", {}).get("title") or [] titles = data.get("message", {}).get("title") or []
return {"status": "pass", "canonical_title": titles[0] if titles else None} return {"status": "pass", "canonical_title": titles[0] if titles else None}
except urllib.error.HTTPError as e: except urllib.error.HTTPError as e:
@ -64,70 +66,27 @@ def check_doi(doi, timeout=20):
return {"status": "skipped", "reason": str(e)} return {"status": "skipped", "reason": str(e)}
def _best_title_match(candidates, title, title_field="title"): def check_title_cross_source(title, timeout=20):
best = max(candidates, key=lambda p: title_similarity(title, p.get(title_field, "") or "")) """Independently re-search by title on a different source (Semantic
sim = title_similarity(title, best.get(title_field, "") or "") Scholar) and require a near-exact title match. This is what catches a
return sim, best.get(title_field) plausible-sounding but entirely invented title/author combination."""
def check_title_cross_source_s2(title, timeout=20):
"""Independently re-search by title on Semantic Scholar and require a
near-exact title match. This is what catches a plausible-sounding but
entirely invented title/author combination."""
try: try:
params = {"query": title, "limit": 3, "fields": "title"} params = {"query": title, "limit": 3, "fields": "title"}
url = f"https://api.semanticscholar.org/graph/v1/paper/search?{urllib.parse.urlencode(params)}" url = f"https://api.semanticscholar.org/graph/v1/paper/search?{urllib.parse.urlencode(params)}"
data = json.loads(fetch(url, timeout=timeout)) with urllib.request.urlopen(url, timeout=timeout) as resp:
data = json.loads(resp.read())
candidates = data.get("data", []) or [] candidates = data.get("data", []) or []
if not candidates: if not candidates:
return {"status": "fail", "reason": "no matching title found on Semantic Scholar"} return {"status": "fail", "reason": "no matching title found on Semantic Scholar"}
sim, matched = _best_title_match(candidates, title) best = max(candidates, key=lambda p: title_similarity(title, p.get("title", "")))
status = "pass" if sim >= 0.9 else "fail" sim = title_similarity(title, best.get("title", ""))
return {"status": status, "similarity": round(sim, 3), "matched_title": matched} if sim >= 0.9:
return {"status": "pass", "similarity": round(sim, 3), "matched_title": best.get("title")}
return {"status": "fail", "similarity": round(sim, 3), "matched_title": best.get("title")}
except Exception as e: except Exception as e:
return {"status": "skipped", "reason": str(e)} return {"status": "skipped", "reason": str(e)}
def check_title_cross_source_openalex(title, timeout=20):
"""Same idea as check_title_cross_source_s2 but against OpenAlex -- a
second, independent source for the title cross-check so a single source
(S2 in particular has real-world rate-limit issues) being unavailable
doesn't drop this whole check to 'skipped' for a batch of candidates."""
try:
params = {"search": title, "per_page": 3, "select": "display_name"}
url = f"https://api.openalex.org/works?{urllib.parse.urlencode(params)}"
data = json.loads(fetch(url, timeout=timeout))
candidates = data.get("results", []) or []
if not candidates:
return {"status": "fail", "reason": "no matching title found on OpenAlex"}
sim, matched = _best_title_match(candidates, title, title_field="display_name")
status = "pass" if sim >= 0.9 else "fail"
return {"status": status, "similarity": round(sim, 3), "matched_title": matched}
except Exception as e:
return {"status": "skipped", "reason": str(e)}
def _combine_title_checks(s2_result, openalex_result):
"""Merge the two independent title cross-checks into one verdict-relevant
check. Coverage differs between S2 and OpenAlex, so one source not
indexing a (real) paper yet is common and must not, by itself, read as
"actively contradicted" the way a single-source design would have taken
it -- a pass from either source is enough; it only counts as an active
fail once neither source could corroborate it and at least one of them
genuinely searched (wasn't just skipped for network reasons)."""
sources = {"semantic_scholar": s2_result, "openalex": openalex_result}
passing = [r for r in sources.values() if r["status"] == "pass"]
if passing:
best = max(passing, key=lambda r: r.get("similarity", 0))
return {"status": "pass", "sources": sources, "similarity": best.get("similarity"),
"matched_title": best.get("matched_title")}
non_skipped = [r for r in sources.values() if r["status"] != "skipped"]
if non_skipped:
return {"status": "fail", "sources": sources,
"reason": "no title match on either Semantic Scholar or OpenAlex"}
return {"status": "skipped", "sources": sources, "reason": "both sources unavailable"}
def verify(title=None, arxiv_id=None, doi=None): def verify(title=None, arxiv_id=None, doi=None):
checks = {} checks = {}
if arxiv_id: if arxiv_id:
@ -135,10 +94,7 @@ def verify(title=None, arxiv_id=None, doi=None):
if doi: if doi:
checks["doi_check"] = check_doi(doi) checks["doi_check"] = check_doi(doi)
if title: if title:
checks["title_cross_source_check"] = _combine_title_checks( checks["title_cross_source_check"] = check_title_cross_source(title)
check_title_cross_source_s2(title),
check_title_cross_source_openalex(title),
)
passed = [c for c in checks.values() if c["status"] == "pass"] passed = [c for c in checks.values() if c["status"] == "pass"]
failed = [c for c in checks.values() if c["status"] == "fail"] failed = [c for c in checks.values() if c["status"] == "fail"]

0
.claude/skills/paper-writing-grounded/SKILL.md → .claude/skills/paper-wiriting-grounded/SKILL.md
View File

BIN
.claude/skills/paper-wiriting-grounded/paper-writing-grounded.skill Normal file
View File

Binary file not shown.

124
CLAUDE.md
View File

@ -1,124 +0,0 @@
# autoPaper — 科研辅助 Claude Skill 库
本仓库不是一篇具体论文的工作区,而是一套持续维护的 Claude Code skill 库,
服务于科研工作流(文献检索 → 核实 → 归档 → 写作)。`references/` 下按主题
归档的文献是这套 skill 实际产出的样例/沉淀,不是本仓库的主体。
## 目录结构
```
.claude/skills/ 所有 skill 的定义,每个子目录一个 skill,只放代码/说明
literature-search-verify/ 检索 + 反幻觉引用核查
SKILL.md skill 说明(frontmatter name 必须与目录名一致)
scripts/ 纯标准库 Python 脚本,无第三方依赖(含 http_utils.py 统一限流重试)
tests/ unittest 单测,全部mock网络请求,不发真实请求
paper-writing-grounded/ 论文写作(强制数据溯源)
SKILL.md
git-commit/ 规范化 commit message 生成与提交
SKILL.md
output/<skill-name>/<topic-slug>/ 各 skill 运行时的草稿区,.gitignore 掉,不进版本库;
按 skill 分子目录、同一 skill 内再按主题分子目录
(topic-slug 规则和 references/ 下的目录名一致),
避免不同 skill、不同主题的草稿互相覆盖/混堆
references/<topic-slug>/ literature-search-verify 归档产出的稳定文献库
references.bib 已验证文献,写作阶段 \cite{} 直接复用这里的 key
README.md 人可读索引,含 suspect/unverified 条目说明
pdfs/ 开放获取的原文 PDF(如有)
```
## 核心设计原则(新增/修改 skill 时必须保持)
1. **反幻觉是硬约束,不是建议**:任何进入正文引用或数字的内容都必须能独立核实
或追溯到真实数据,验证不通过就必须显式标记(`suspect`/`unverified`/
`[需要数据: ...]`),绝不能为了让输出"看起来完整"而悄悄丢弃或蒙混过关。
2. **脚本优先于临场编 API 调用**:能写成 `scripts/` 里可重复运行的脚本就不要
指望模型每次现场拼 HTTP 请求——后者不可复现、容易在细节上出错。脚本只用
标准库,不引入第三方依赖,保证任何环境下拿来就能跑。
3. **草稿区与交付物分离**:草稿(项目根目录下的 `output/<skill-name>/<topic-slug>/`)
只是运行痕迹,不是可信的最终产物,也不放在 `.claude/skills/` 里面(那里只放
skill 代码本身);确认稳定后要显式归档到 `references/<slug>/` 这类项目级
目录,才算数。同一 skill 下不同主题的草稿必须分子目录,不能用无区分度的
通用文件名（如`t1.json`）散落在同一层——这类命名冲突曾经真实发生过。
4. **skill 之间通过约定(如 bibtex key)解耦协作**,而不是互相读对方内部状态;
每个 SKILL.md 末尾应有一节说明它和其他 skill 的配合方式。
5. **目录名必须和 SKILL.md frontmatter 里的 `name:` 完全一致**——skill 发现/
调用是按目录名走的,两者不一致会导致"文档里写的名字"和"实际能唤起的名字"
对不上(曾经出现过 `paper-wiriting-grounded` 目录名手滑打错、和 frontmatter
里正确拼写的 `paper-writing-grounded` 不一致的问题,已修正)。
## 已知薄弱环节 / 待办方向
- 目前只覆盖"检索/核实"与"写作/溯源"两段,引用一致性核查(定稿里的
`\cite{}` 是否都对得上 `references.bib`、有没有归档了却从未引用的条目)
还没有对应 skill。
- `search_openalex.py` 目前只用了 OpenAlex Works API 里比较基础的字段
(标题/作者/年份/venue/DOI/引用数/开放获取PDF),没有用它的 concept/topic
分类字段做更细的学科过滤——如果以后要精确限定学科方向,这是可以深挖的点。
## 下一步计划
1. ~~给核心脚本补单测 + 给网络请求加限流重试~~ **已完成**:新增
`scripts/http_utils.py` 统一封装限流重试(429/5xx指数退避,遵守
`Retry-After`),`search_arxiv/crossref/semantic_scholar/openalex.py` 和
`verify_citation.py` 都已接入;`verify_citation.py` 的跨源标题核查改成
同时查 Semantic Scholar + OpenAlex 两个独立源(任一命中即通过),不再
单点依赖 S2;新增 `search_openalex.py` 作为第四个检索源;`archive_references.py`
的 `slugify()` 修了中文主题名被折叠成通用"references"名的问题;
`scripts/tests/` 下补了 37 个 unittest 单测(全部mock,不发真实请求),
覆盖上述所有改动。
2. **新增"引用一致性核查" skill**(`citation-consistency-check`):检查定稿里
所有 `\cite{}` 的 key 是否都能在对应主题的 `references/<slug>/references.bib`
里找到,以及有没有已归档但从未被正文引用的"僵尸条目",在
literature-search-verify 和 paper-writing-grounded 之间补上这个校验环节。
3. **新增"图表生成" skill**(`figure-from-data`,独立 skill,不并入
paper-writing-grounded):把全局 dataviz skill 的画图能力和本项目"数据不
能编"的红线结合起来——图上出现的每一个数字/误差棒/显著性标注都必须能
追溯到用户提供的真实数据,追溯不到就必须显式标记,不能为了图好看而插值/
编造。用户目前"才刚开始"做实验,这个和 reproducibility-checklist 一起
算当前阶段实际用得上的。
4. **新增"实验可复现性自查" skill**(`reproducibility-checklist`):检查一个
实验结果和"跑出这个结果的环境/随机种子/超参数/代码版本"之间是否可追溯,
在博士研究早期建立这个记录习惯,比后期(投稿前)才补收益更大。具体检查
项和触发时机(每次跑完实验?还是准备写进论文前?)还需要进一步讨论确定。
5. **新增"投稿格式合规检查" skill**(`submission-checklist`):投稿前检查页数
限制、双盲匿名化处理、模板合规、supplementary材料要求等——失败代价直接
(格式不合规可能直接被拒),但用户目前还没到投稿周期,优先级低于上面几项,
先记录设想,不着急做。
6. **新增"审稿意见回复" skill**(`rebuttal-writing-grounded`):和
paper-writing-grounded 共享同一条红线——不能为了让回复显得更有说服力,
而承诺做不到的新实验或编造补充结果;但写作场景(逐条对应审稿人意见、
语气要求"礼貌但坚定")不同,应该做成姊妹 skill 而不是塞进
paper-writing-grounded 里。用户目前还没进入投稿/答辩周期,优先级最低,
先记录设想。
7. **新增"实验设计常见陷阱清单" skill**(`experimental-design-checklist`,
优先级低、置信度低):不强制流程,只是一份"容易漏掉的检查项"提醒(有没有
设基线/消融、统计检验方法选得对不对、有没有偷偷用测试集调过参),因为
"实验设计得好不好"本质是统计学/领域判断力,skill 只能提醒别漏掉常见坑,
不能替用户判断设计是否合理——做的时候要非常克制,避免让用户误以为"清单
过了=设计没问题"。
8. **新增"组会/答辩/会议报告大纲" skill**(`presentation-outline`,优先级低、
价值存疑):把已有的真实结果整理成报告大纲,同样要遵守"不编内容"的红线。
价值有限,因为这类大纲高度依赖听众和场合,通用流程能提供的帮助有限;暂
不确定要不要做,先记录设想。
9. **新增"导师进展汇报" skill**(`advisor-progress-report`,优先级低、价值
存疑):从 git log / 实验记录整理成给导师的周报。风险是"总结不当会歪曲
实际进度"——如果做,必须严格限定为"只整理已确认的事实,不做主观进度
评估",且这类沟通策略本身因人而异,通用 skill 能提供的价值可能不大。
10. **讨论过但决定不做**:
- "新颖性/查重式文献扫描"(检索这个想法是否已被做过)。结论是它和
literature-search-verify 的检索底层高度重合,而"够不够新颖"本质是
判断力问题,skill 顶多能帮忙把相关已有工作找全,这部分
literature-search-verify 已经能覆盖大半,边际价值不足以单独立项。
- "研究问题提出/创新点判断"。这是纯判断力/领域洞察力问题,不适合
productize——skill 最多能辅助"检索现有工作看有没有人做过"(即上一条
讨论过的新颖性扫描),但"这个问题值不值得做"必须是人的判断,做成
强流程 skill 反而有"流程走完=判断没问题"的误导风险。
- 以上两条如果以后发现实际需求很明确,可以重新评估这些决定。
## 新增 skill 时的约定
- SKILL.md 的 frontmatter `description` 要写清楚"什么场景下必须触发这个
skill",因为这是 skill 被自动选中的唯一依据。
- 正文按"为什么需要 / 工作流程 / 和其他 skill 的配合"组织,和现有两个 skill
保持同样的结构和颗粒度。
- 目录名 = frontmatter `name`,不要有拼写差异。

2
references/uav_aeromagnetic_compensation/README.md
View File

@ -53,4 +53,4 @@ Direction covered: UAV/airborne aeromagnetic compensation (Tolles-Lawson family)
Queried via arXiv + Crossref + Semantic Scholar (S2 was rate-limited (HTTP 429) for much of the session, so most entries only got single-channel verification -- arXiv ID or DOI resolution -- rather than the additional cross-source title check; this is noted per-entry as "unverified"/"skipped" in the raw JSON reports, not silently upgraded to double-verified). Queried via arXiv + Crossref + Semantic Scholar (S2 was rate-limited (HTTP 429) for much of the session, so most entries only got single-channel verification -- arXiv ID or DOI resolution -- rather than the additional cross-source title check; this is noted per-entry as "unverified"/"skipped" in the raw JSON reports, not silently upgraded to double-verified).
Searched but found no on-topic hits: Kalman-filter-based aeromagnetic compensation; genetic-algorithm/PSO-based aeromagnetic compensation. Searched but found no on-topic hits: Kalman-filter-based aeromagnetic compensation; genetic-algorithm/PSO-based aeromagnetic compensation.
Not covered at all: CNKI/Wanfang/VIP (Chinese databases, no public API) -- use the Zotero Connector browser extension logged into a university account for these. Not covered at all: CNKI/Wanfang/VIP (Chinese databases, no public API) -- use the Zotero Connector browser extension logged into a university account for these.
Raw per-query JSON search/verification reports (including filtered-out noise from ambiguous keyword matches like "Tolles"/"Lawson" as surnames) are kept in output/literature-search-verify/uav_aeromagnetic_compensation/ for audit purposes and are not part of this archive. Raw per-query JSON search/verification reports (including filtered-out noise from ambiguous keyword matches like "Tolles"/"Lawson" as surnames) are kept in .claude/skills/literature-search-verify/output/ for audit purposes and are not part of this archive.