Building a retrieval-augmented generation system over clinical guidelines is not a retrieval problem. It is a trust problem. When a Licensed Clinical Doctor queries a system for DSM-5-TR diagnostic criteria, a hallucinated criterion is not a UX failure. It is a patient safety event. This article covers the practical architecture decisions that separate a compliant, citation-faithful clinical RAG system from a general-purpose chatbot bolted onto a PDF reader.
At TheraPetic® Healthcare Provider Group, our engineering team has spent considerable time building the document intelligence layer behind HANK AI and the intake screening pipeline at mypsd.org. The patterns described here reflect real production decisions, not whiteboard theory. The focus keyword throughout is clinical guideline RAG, and every architecture decision flows from what makes that specific use case different from enterprise search or legal discovery.
Why Standard LLMs Fail Clinical Guideline Retrieval
Large language models trained on web-scale data absorb clinical language well. They can write plausibly structured diagnostic summaries. That plausibility is the hazard.
DSM-5-TR criteria are precise. The distinction between a major depressive episode requiring five symptoms from a specific cluster and a persistent depressive disorder with two symptoms from a different cluster is not negotiable. A model that has seen thousands of paraphrased summaries of these criteria will generate confident, syntactically correct, clinically wrong output when asked to reproduce verbatim diagnostic thresholds.
Research published in JAMA Internal Medicine and covered in NEJM AI has consistently documented that GPT-class models produce plausible but unfaithful clinical text when asked to reproduce guideline content from memory. The problem is not intelligence. It is that parametric memory compresses, merges and interpolates. Clinical guidelines require exact retrieval.
RAG addresses this by grounding generation in retrieved source chunks. But clinical guideline RAG introduces constraints that general RAG tutorials do not address: versioning requirements (DSM-5 vs DSM-5-TR), structured criterion numbering, cross-references between sections, and the regulatory need to trace every generated claim back to a citable source passage.
Corpus Preparation and Metadata Anchoring
The corpus preparation phase determines whether your retrieval layer can honor citation fidelity requirements. Most RAG tutorials skip this step or treat it as "just ingest your PDFs." For clinical guidelines, that approach fails at the first query.
The three primary corpora that TheraPetic®'s clinical NLP team works with are the DSM-5-TR (American Psychiatric Association), NICE clinical guidelines for mental health conditions, and APA clinical practice guidelines. Each has structural properties that require deliberate preprocessing.
Every chunk that enters the vector store must carry structured metadata at minimum:
- Source document identifier and version (e.g., DSM-5-TR, 2022 revision)
- Section number and subsection heading using the source document's own numbering
- Page range from the canonical print edition
- Content type tag: diagnostic criteria, differential considerations, specifier definitions, or epidemiological notes
- A SHA-256 hash of the original chunk text to detect drift between ingestion runs
This metadata layer is what makes citation fidelity possible downstream. Without it, your system can retrieve the right text and generate a response that says "according to the DSM-5-TR" but cannot point to a section number that a clinician can verify. That is not good enough for clinical use.
Chunking Strategies for Long Clinical Documents
Chunking clinical guidelines requires domain-aware segmentation, not character-count splitting. This is where clinical guideline RAG diverges most sharply from standard enterprise RAG.
The DSM-5-TR diagnostic criteria sections are themselves atomic units. Criterion A for Major Depressive Disorder contains nine sub-items and must be retrieved as a single unit. Splitting it at 512 tokens mid-criterion produces chunks that are semantically incomplete. A retrieval system that returns half of Criterion A and generates a response from it will miss required symptoms. The generated output will be fluent and wrong.
The chunking strategy that our engineering team recommends for DSM-5-TR-class documents has three tiers.
The first tier is the criterion block. Each labeled criterion set (Criterion A, Criterion B, etc.) is a non-divisible chunk regardless of token length. If a criterion block exceeds your embedding model's context window, the solution is not to split it. The solution is to use a model with a longer context window or to store the full block with a pointer structure that retrieves it as a single unit.
The second tier is the differential diagnosis section for each disorder. These sections contain cross-references to other disorders and must be chunked to preserve the relational structure. Strip the prose but retain the disorder name anchors as metadata tags so that similarity search can surface differential content when a query pattern matches multiple disorders.
The third tier is supporting prose: specifier definitions, prevalence data and associated features. These can tolerate sentence-level chunking with 10 to 15 percent overlap, and they benefit from semantic chunking algorithms rather than fixed-size windows. The LlamaIndex semantic splitter and similar boundary-detection approaches outperform fixed-window chunking on this tier because guideline prose already uses consistent topic-sentence structure.
NICE guidelines introduce a different challenge. They publish in HTML with recommendation grades (A, B, GPP) embedded inline. The grade is part of the clinical meaning. A Grade A recommendation backed by randomized controlled trial evidence carries different authority than a Good Practice Point. Your chunking pipeline must extract the grade as metadata, not treat it as incidental prose. Downstream grounding checks can then surface recommendation strength as part of the citation trace.
Retrieval Pipeline Design and Grounding Scores
A production clinical guideline RAG pipeline needs at least three retrieval passes before generation begins.
The first pass is dense vector retrieval using a biomedical embedding model. General-purpose embeddings trained on web text perform poorly on clinical terminology because they were not trained to distinguish diagnostic terms from their lay equivalents. BioLORD and clinical fine-tunes of sentence-transformers trained on MIMIC or PubMed abstracts outperform ada-002 class embeddings on medical semantic similarity benchmarks. The embedding choice is not a minor configuration decision. It determines recall at the criterion level.
The second pass is sparse keyword retrieval using BM25 over the structured metadata fields. Dense retrieval misses exact criterion numbers. If a clinician asks about "Criterion B for Persistent Depressive Disorder," dense retrieval may surface semantically adjacent content from Major Depressive Disorder instead. BM25 over the section number and disorder name fields corrects this. The two signals are fused using Reciprocal Rank Fusion before passing the candidate set to the reranker.
The third pass is cross-encoder reranking. The reranker scores each candidate chunk against the original query for relevance. In clinical contexts, the reranker should be fine-tuned on clinical question-answer pairs rather than using a general MSMARCO-trained model. The output of this pass is the final grounded context window passed to the generation model.
Grounding scores are computed at retrieval time, not generation time. Each chunk that enters the context window receives a grounding confidence value based on the reranker score. This value is stored alongside the generated response. If no chunk in the retrieved set exceeds a configurable confidence threshold, the system must be designed to return a refusal or escalation rather than generate from low-confidence context. In clinical settings, a confident wrong answer is worse than a transparent non-answer.
Citation Fidelity Requirements in Clinical NLP
Clinical guideline RAG has a citation fidelity requirement that has no parallel in enterprise search: every generated claim must be traceable to a specific passage that a licensed professional can locate and read in the source document.
This means citation fidelity is an architecture constraint, not a prompt engineering aspiration. The system cannot hope that the model will cite correctly. It must be structured so that the model can only cite what was retrieved.
The implementation pattern that satisfies this constraint is span-level attribution. After generation, a post-processing layer aligns each sentence or clause in the generated output against the retrieved chunks using a natural language inference (NLI) model fine-tuned for entailment. Sentences that are not entailed by any retrieved chunk are flagged as grounding failures. The NLI model does not need to be large. A BERT-base model fine-tuned on MedNLI performs adequately for this attribution task and adds minimal latency.
The citation format presented to end users at TheraPetic®-affiliated interfaces includes the document name, section number, and a verbatim pull-quote of the supporting passage. The verbatim pull-quote is critical. It allows a Licensed Clinical Doctor reviewing the output to verify that the cited passage actually supports the generated claim. "According to DSM-5-TR" without a page or section reference is not citation fidelity. It is citation appearance.
Hallucination Mitigation With Grounding-Based Guardrails
Hallucination in clinical guideline RAG takes two forms that are both dangerous and easy to miss.
The first form is fabricated criteria: the model generates a symptom threshold or duration requirement that does not exist in the source document. This is the form most people think of when they think of hallucination. It is detectable with the NLI attribution layer described above.
The second form is accurate-but-misattributed content: the model retrieves correct information from one disorder's criteria section and presents it as applying to a different disorder. This form is harder to detect because the NLI model may find entailment with the retrieved chunk without flagging that the chunk was retrieved for the wrong disorder. Metadata filtering at retrieval time is the primary mitigation. When a query is resolved to a specific disorder by a named entity recognition step upstream, retrieval should be constrained to chunks tagged with that disorder's identifier.
The grounding score threshold policy needs to be explicit and documented. At TheraPetic®, the policy for the HANK AI clinical intake layer is a hard cutoff: if the maximum grounding score for any retrieved chunk falls below 0.72 on the reranker scale, the response is suppressed and the query is routed to a human clinical reviewer. This threshold was calibrated against internal evaluation sets and reviewed by our Licensed Clinical Doctors. The number is not universal. Your threshold depends on your embedding model, reranker architecture and the risk tolerance of your clinical use case.
Prompt-level guardrails complement but do not replace grounding-based controls. System prompts that instruct the model to cite only retrieved passages and refuse to answer from memory reduce hallucination frequency but do not eliminate it. Instruction-following models can still violate retrieval-only constraints under adversarial prompting or when the retrieved context is sparse. Architecture-level controls (grounding scores, NLI attribution, metadata filtering) are not optional. They are the floor.
TheraPetic® Infrastructure Patterns and Lessons Learned
The clinical guideline RAG system that supports intake screening at verify.mypsd.org went through three major architecture iterations before reaching production stability. The lessons from those iterations are worth documenting.
The first lesson is that document versioning is a maintenance problem, not a one-time setup. The DSM-5-TR superseded the DSM-5 in 2022. NICE updates individual guidelines on rolling schedules. APA practice guidelines are revised on irregular cycles. A clinical RAG system that does not have an automated pipeline for detecting source document updates and re-ingesting affected chunks will drift out of date silently. Clinicians querying the system will not know they are receiving guidance from a superseded edition. Version tagging and automated update checks against source publishers are not optional infrastructure.
The second lesson is that evaluation sets must be built by Licensed Clinical Doctors, not by engineers. The evaluation set for our DSM-5-TR retrieval layer was constructed by reviewing a random sample of 200 queries from early internal testing. For each query, a Licensed Clinical Doctor on our team identified the correct source passage and wrote the expected citation. Engineer-written evaluation sets consistently underweight edge cases around differential diagnosis boundaries and specifier applicability. Clinical expertise in the evaluation loop is what makes the grounding threshold meaningful.
The third lesson is that HIPAA Safe Harbor deidentification must be applied before any query text from real clinical interactions enters the retrieval pipeline logs. Under HIPAA Safe Harbor requirements, 18 categories of patient identifiers must be removed before data can be stored or processed outside a covered entity's protected environment. Query logs for a clinical RAG system are not exempt. The MyDataKey data governance layer that TheraPetic® uses for deidentification applies a rule-based scrubber followed by a NER model to catch identifiers that pattern rules miss. This is not a privacy best practice. It is a legal requirement.
The fourth lesson is about transparency with end users. Every response generated by HANK AI that draws on DSM-5-TR or NICE guideline content displays the retrieved source passage to the Licensed Clinical Doctor reviewing the intake summary. The system does not present generated summaries as authoritative clinical conclusions. It presents them as structured summaries grounded in retrieved guideline text, with the retrieved text visible for verification. This transparency design is what allows Licensed Clinical Doctors to trust and override the system. AI that hides its reasoning from clinicians does not earn clinical trust. AI that shows its work does.
Clinical guideline RAG is one of the highest-stakes applications of retrieval-augmented generation in use today. The architecture decisions around chunking, embedding model selection, grounding score policy and citation attribution are not implementation details. They are patient safety decisions. Build them that way.
For more on TheraPetic®'s clinical AI infrastructure and the HANK AI platform, visit therapetic.net or explore the service dog verification tooling at servicedog.ai.