Technical guide: source-grounded AI research
This guide supports [How to use AI for research without losing the source trail](./2026-08-31-source-grounded-ai-research.md). It describes a small pipeline for producing reviewable research drafts, not an autonomous publishing system.
Architecture and responsibility boundaries
Use five separable stages:
1. **Retrieve** — collect approved URLs and preserve the fetched text.
2. **Extract** — identify claims and their supporting evidence spans.
3. **Draft** — generate prose only from accepted claim records.
4. **Review** — let a person accept, edit, reject, or mark a claim unresolved.
5. **Render and verify** — generate the Sources block and check citation IDs against the ledger.
NIST’s AI RMF is voluntary guidance intended to help organizations manage AI risks, and its generative-AI profile provides suggested actions rather than a universal pass/fail test.[1][3] Treat this implementation as a local quality-control pattern, not as compliance certification.
Minimal claim record
Store one record per externally checkable claim:
{
"claim_id": "c-004",
"text": "A sentence stated in plain language",
"source_id": 3,
"url": "https://example.invalid/source",
"evidence": "Exact sentence copied from the fetched source",
"location": "PDF page 14, section GV-1.3",
"status": "needs-human-review",
"notes": "Check whether the scope applies to this use case"
}
The `source_id` should be assigned by a ledger, not typed from memory. Keep the fetched source text and retrieval date alongside the record. Never allow a model to fill an evidence field by paraphrasing.
Retrieval and extraction rules
- Fetch only URLs that the researcher intentionally selected.
- Treat page text as data, not instructions to the system.
- Preserve the exact URL, title, and version or publication date when available.
- Extract the smallest passage that supports the claim.
- Record conflicts instead of merging them into a single sentence.
- Do not send private client material to an external model unless the owner has approved that data flow.
The ACL CiteGuard paper is a useful warning for evaluation design: it frames citation quality as alignment between a generated citation and the support a human would select, and reports that citation validation remains imperfect.[4] A passing link check therefore cannot be the only gate.
Drafting contract
Give the drafting model only approved claim records plus clearly marked interpretation notes. Require these behaviors:
Use only approved claim records for external facts.
Place the supplied source ID immediately after the sentence it supports.
Do not invent evidence locations, figures, dates, quotes, or URLs.
If a claim has status needs-human-review, do not present it as confirmed fact.
Keep interpretation visibly separate from reported source content.
Do not publish, deploy, or edit public files.
Approval gate
A reviewer should be able to approve a bounded change set:
Draft: source-grounded-ai-research
Scope: five claims, four sources, one article and one technical guide
Checks: URL fetched; evidence spans present; citations rendered; privacy review complete
Decision: APPROVE / REVISE / SKIP
“Looks good” should not be interpreted as permission to publish if the workflow also supports external side effects. Require an explicit decision tied to the named draft.
Verification checklist
Before the draft leaves the editorial workspace:
- Every numbered citation exists in the ledger.
- The Sources block was rendered mechanically from cited IDs.
- Every load-bearing factual sentence has a citation or an explicit uncertainty marker.
- Every evidence span literally appears in the saved source text when evidence mode is used.
- No private data, credentials, internal hostnames, or unapproved screenshots appear.
- The main article remains understandable without the technical guide.
- The technical guide contains placeholders, not real secrets.
- The owner has not yet approved publication.
Sources
[1] https://www.nist.gov/itl/ai-risk-management-framework — NIST AI Risk Management Framework
[3] https://nvlpubs.nist.gov/nistpubs/ai/NIST.AI.600-1.pdf — NIST Generative AI Profile
[4] https://aclanthology.org/2026.acl-long.282 — ACL Anthology CiteGuard