Find Paper References

---
name: find-paper-references
description: Automatically find references for academic paper Markdown files. Reads full paper text, identifies each knowledge point requiring citation (epidemiological data, mechanism descriptions, existing research conclusions, etc.), searches PubMed for 3-5 most relevant articles per kn...
license: MIT
author: AIPOCH
---
> **Source**: [https://github.com/aipoch/medical-research-skills](https://github.com/aipoch/medical-research-skills)


# find-paper-references — Academic Paper PubMed Reference Finder

## Tools

Scripts are located in the `scripts/` subdirectory of this skill:

- `batch_search.py` — **Main entry point**: accepts all queries at once, parallel search, batch esummary fetch, typical runtime 10-20s
- `remap_refs.py` — Converts `[PMID:XXXXXXXX]` markers to `[1][2]` numbering and generates formal references (**not called by this skill — handled by subsequent endnote/zotero skill**)
- `convert_to_docx.py` — Converts `.md` to `.docx` (**not called by this skill**)

### Skill Boundary

```
┌─────────────────────────────────────────────────────┐
│  find-paper-references                              │
│  ─────────────────────────────────────────────────  │
│  Step 0-3: Identify knowledge points → Build PubMed search JSON │
│  Step 4:   batch_search.py batch search                         │
│  Step 5:   Select articles → Insert [PMID:xxxxxxxx] markers     │
│  Step 6:   Write to .md file                                    │
│  Step 7:   Generate _candidates.md candidate reference list     │
│  Step 8:   Prompt user to choose EndNote / Zotero to continue   │
│  ──────────────── Workflow ends ──────────────────────────────── │
│                                                     │
│  Formatting → format-references-endnote (EndNote)              │
│            → format-references-zotero (Zotero)                 │
└─────────────────────────────────────────────────────┘
```

---

## Workflow (Execute in Order)

### Step 0: Ask for NCBI API Key

Before starting any search, check and ask:

```
Do you have an NCBI API key? If so, search speed increases from 3 req/s to 10 req/s.
No worries if not — just skip and we'll proceed.
```

- **User provides key**: Before all script calls in this session, execute:
  ```bash
  # Windows
  $env:NCBI_API_KEY = "user_provided_key"
  # Mac/Linux
  export NCBI_API_KEY="user_provided_key"
  ```
  Then continue the workflow normally. Key is valid only for this session, not written to any file.

- **User doesn't have one / skips**: Proceed to Step 1. The script automatically uses the conservative 3 req/s rate.

### Step 1: Determine Target File + Article Type

Read the full file content, then determine article type — can be auto-detected or confirmed with the user:

**Auto-detection rules:**
- Contains `## Materials and Methods` / `## Methods` section → **Research Article**
- No such section, body consists mainly of review-style paragraphs → **Review**
- When uncertain, ask the user directly

---

## Article Type Citation Rules

Based on the detected type, this workflow must follow the corresponding rules:

### Research Article Rules
```
Citation zones:  Introduction and Discussion only
            Materials/Methods and Results → skip entirely, insert no citations
Per knowledge point: max 2 articles (top 2 by relevance, no padding)
Total target: at least 30 unique PMIDs in the text
```

### Review Rules
```
Citation zones:  All paragraphs except Conclusion
            Conclusion paragraphs → skip
Per knowledge point: max 5 articles
Total target: approximately 1 citation per 100 words, +/- 20%
            Target range = [word_count*0.8/100, word_count*1.2/100] (rounded)
            Example: 5000 words → target 40-60 articles; 8000 words → target 64-96 articles
```

---

### Step 2: Identify Sentences Requiring Citations

Read through the paper, **strictly following the citation zone rules for the current article type**, skip sections that don't need citations.

Within allowed citation zones, identify all **knowledge points requiring literature support**:

**Needs citation (applicable to allowed sections):**
- Epidemiological data ("Lung cancer is one of the most common malignancies worldwide")
- Known biological functions of proteins/genes ("GPX4 reduces lipid peroxides to non-toxic alcohols")
- Signaling pathway descriptions ("xCT mediates cellular cystine uptake")
- Published research conclusions ("TRIM3 is downregulated in breast cancer")
- Known mechanisms of treatments/drugs ("Erastin induces ferroptosis by inhibiting xCT")

**Never needs citation:**
- This study's own experimental results
- Instrument/reagent descriptions in Methods (entire section skipped for Research Articles)
- Results section (entirely skipped for Research Articles)
- Conclusion section (skipped for Reviews)
- General technical operation descriptions

**After identification, estimate total knowledge points:**
- Research Article: target ≥ 30 unique citations → typically need to identify 20-35 knowledge points
- Review: **calculate total word count first**, estimate target citations at 1 per 100 words, typically need knowledge points close to target number (1-2 articles per point)

Organize identified knowledge points into a list, each containing:
- Section name (to confirm it's in an allowed citation zone)
- The sentence from the original text
- Core concept of the knowledge point (for generating search terms)

### Step 3: Organize All Queries into Temporary JSON

Organize all knowledge points into the following format, write to `%TEMP%\ref_queries.json` (or `/tmp/ref_queries.json`):

```json
[
  {"id": 1, "description": "global lung cancer incidence", "query": "global cancer statistics 2020 GLOBOCAN lung cancer Sung", "method": "pubmed"},
  {"id": 2, "description": "ferroptosis definition", "query": "Ferroptosis iron-dependent nonapoptotic cell death Dixon 2012", "method": "pubmed"},
  {"id": 3, "description": "TRIM3 structure and function", "query": "TRIM3 ubiquitin ligase RING domain substrate degradation", "method": "litsense"},
  ...
]
```

**Method options:**
- `"pubmed"` — Keyword-based precise search (recommended for most knowledge points)
- `"litsense"` — Semantic search (suitable for descriptive sentences, specific protein functions, etc.)
- `"auto"` — Try LitSense first; if results < 2 articles, automatically fall back to PubMed

**Query writing guidelines:**
- Use English, 3-8 keywords
- Prefer MeSH terms (ferroptosis, non-small cell lung cancer, ubiquitin ligase)
- For known classic papers, add author or year to query ("Dixon 2012", "Stockwell 2017")

### Step 4: Run batch_search.py Once to Get All Results

```bash
python "SKILL_DIR/scripts/batch_search.py" "%TEMP%\ref_queries.json" --max 5
```

Script searches all queries in parallel, batch-fetches esummary, **typical runtime 10-20 seconds** (vs. minutes for sequential searching).

Output JSON format:
```json
[
  {"id": 1, "description": "...", "results": [{pmid, title, authors, journal, year, doi}, ...]},
  ...
]
```

**If any result is empty**, create a new JSON file with adjusted keywords and re-run batch_search.py.

### Step 5: Select Most Relevant Articles and Insert PMID Markers

For each sentence requiring citation:

1. Sort candidates by relevance, select according to **article type limits**:
   - **Research Article**: max **2** per knowledge point, select only the most relevant
   - **Review**: max **5** per knowledge point, can cover different research angles
2. Insert `[PMID:XXXXXXXX]` at **end of original sentence**; for multiple: `[PMID:111][PMID:222]`
3. If same PMID already cited, reuse it — do not insert duplicates

**After insertion, count current unique PMIDs:**
- Research Article: if fewer than 30, return to Step 2 to identify more knowledge points
- Review: **calculate word count, compute target range [words*0.8/100, words*1.2/100] (rounded)**, if not within range, return to Step 2 to add or reduce

**Example (before/after):**
```
Before: Ferroptosis is a form of regulated cell death driven by iron-dependent lipid peroxidation.
After:  Ferroptosis is a form of regulated cell death driven by iron-dependent lipid peroxidation[PMID:25789077].
```

**Flexible formats supported by remap_refs.py (all handled correctly):**
- Case insensitive: `[PMID:123]` `[pmid:123]` `[Pmid:123]`
- Spaces around colon: `[PMID: 123]` `[ PMID : 123 ]`
- Multiple in one bracket: `[PMID:123, PMID:456]` → `[1][2]` (spaces and comma variants accepted)

### Step 6: Write PMID-Marked Full Text to New File

**Do not overwrite the original file.** Write the full text with `[PMID:XXXXXXXX]` markers to a **new file**:

- Original `paper.md` → New file `paper_refs.md` (generated in same directory)

Naming rule: remove `.md` suffix from original filename, add `_refs.md`. Examples:
- `TRIM3_ferroptosis.md` → `TRIM3_ferroptosis_refs.md`
- No extension case: `manuscript` → `manuscript_refs.md`

Keep the original file unchanged — no modifications.

### Step 7: Write Candidate References to Separate File

**Do not** append the reference candidate section to the original `.md` file — it will interfere with subsequent formatting.

Instead, generate a separate candidate file `originalfilename_candidates.md` in the same directory:

```markdown

## Reference Candidates

> Below are PubMed search results for each knowledge point. Those selected for the text are marked with checkmark.
> Next step: Use format-references-endnote or format-references-zotero skill for formatting.

### Knowledge Point 1: [brief description]
- ✓ **[PMID:25789077]** Dixon SJ et al. "Ferroptosis: an iron-dependent form of nonapoptotic cell death." *Cell* 2012;149(5):1060-72.
- [PMID:26593993] Stockwell BR et al. "Ferroptosis: a regulated cell death nexus linking metabolism, redox biology, and disease." *Cell* 2017;171(2):273-285.
- [PMID:31634899] ...

### Knowledge Point 2: [brief description]
...
```

**Step 7 completion ends this workflow.** Do not run remap_refs.py or convert_to_docx.py.

### Step 8: Proactively Remind User to Choose Formatting Tool

After Step 7, **must proactively ask the user** to choose next step:

```
All citation markers have been inserted (N unique references total). Ready to format and export to Word?

Do you use EndNote or Zotero for reference management?
```

- User chooses **EndNote** → immediately load `format-references-endnote` skill to continue
- User chooses **Zotero** → immediately load `format-references-zotero` skill to continue
- User says "not now" / "let me check first" → inform them "say 'format references' anytime to continue"

**Do not** end silently — this reminder is mandatory.

---

## Next Step: Reference Formatting (Separate Skill)

After the `[PMID:xxxxxxxx]`-marked .md file is generated, choose based on user's reference manager:

### EndNote Users

Use **format-references-endnote** skill. It will:
1. Convert `[PMID:xxxx]` markers to EndNote CWYW-recognized `{Author, Year, Title}` placeholders
2. Generate `.ris` file for automatic import to local EndNote library
3. Output `.docx` file — user clicks "Update Citations" once in Word to complete formatting


### Zotero Users

Use **format-references-zotero** skill. It will:
1. Convert `[PMID:xxxx]` markers to native Zotero field codes
2. Search/download any CSL citation style
3. Output `.docx` file — click "Refresh" once in Word to complete

---

## NCBI API Key (Optional, Free)

NCBI provides free API keys. With a key, rate limit increases from 3 req/s to 10 req/s, concurrent workers increase from 4 to 8, making searches faster and less likely to trigger rate limiting.

**How to get one:**
1. Register NCBI account: https://www.ncbi.nlm.nih.gov/account/
2. Go to Account Settings → API Key Management → Generate

**How to use:**
```bash
# Temporary (current session only)
set NCBI_API_KEY=your_key    # Windows
export NCBI_API_KEY=your_key  # Mac/Linux

# Permanent (recommended: add to system env vars or .env file)
```

**Works fine without a key** — script automatically uses conservative 3 req/s rate and auto-retries on 429 errors.

## Important Notes

- Search terms must be in English (PubMed indexes primarily English literature)
- For very recent research (2024 onwards), PubMed indexing may have delays
- On 429 rate limiting, script auto-retries with backoff (up to 4 times) — no manual intervention needed

## Input Validation

This skill accepts requests that match the documented purpose of `find-paper-references` and include enough context to complete the workflow safely.

Do not continue the workflow when the request is out of scope, missing a critical input, or would require unsupported assumptions. Instead respond:

> `find-paper-references` only handles its documented workflow. Please provide the missing required inputs or switch to a more suitable skill.