What AI actually cites when you ask a medical question
Ask a general-purpose AI chatbot a medical question — something clinical, something you might genuinely need the answer to — and look at the citations. Not the answer itself, which will sound fluent and confident. Look at where it claims the answer comes from.
Analyses of AI chatbot citations in health-related queries reveal a pattern that should alarm any clinician. Nearly a third of all citations point to health media websites — hospital blogs, wellness portals, health news sites. These are websites that write about medical research, not websites that publish medical research. Another quarter of citations come from commercial and affiliate-driven websites — pages selling supplements, promoting clinics, or running health content as a vehicle for advertising revenue.
Actual academic and research sources — the peer-reviewed journals, the randomised controlled trials, the systematic reviews, the clinical practice guidelines — make up less than a quarter of all citations. PubMed Central, the single largest repository of open-access biomedical research, accounts for a fraction of one percent.
When you ask a general AI a medical question, it cites health blogs and commercial websites more often than it cites peer-reviewed research. The sources it finds are overwhelmingly open-access web content, not the evidence base clinicians actually use.
Why the citations are wrong: the open web problem
This is not a bug in the AI. It is a structural consequence of how general-purpose AI systems find information. These systems search the open web — the same web that Google searches. And the open web has a fundamental bias: content that is freely accessible, search-engine-optimised, and frequently linked ranks higher than content that is paywalled, jargon-dense, and cited in academic bibliographies.
The full text of a randomised controlled trial published in the Journal of Urology sits behind a paywall. The AI cannot access it. The blog post that some health media site wrote about that trial — summarising the abstract, adding stock photos, and stuffing keywords for SEO — is freely available and ranks well in web search. So when the AI looks for information about that trial, it finds the blog post, not the paper.
This is true at scale. The vast majority of peer-reviewed medical literature is not freely accessible on the open web. It lives behind publisher paywalls (Elsevier, Springer, Wiley), in institutional repositories, or in subscription databases. What IS freely accessible is the ecosystem of secondary content that has grown up around academic publishing: hospital marketing blogs, patient information pages, health news aggregators, continuing education portals, and pharmaceutical company websites.
The AI is not choosing bad sources out of negligence. It is choosing the only sources available to it. And in medicine, the freely available sources are systematically less reliable than the paywalled ones. This is the opposite of most domains, where open access generally correlates with transparency. In academic medicine, the most rigorous evidence is behind the highest walls.
The AI cites blog posts because that is what it can see. The study with the methods, the results, the confidence intervals, and the nuance is behind a paywall. The simplified summary written for patients — or for ad revenue — is not.
The layers of citation failure
The citation problem with general AI is not a single failure mode — it is a cascade of failures, each worse than the last. Understanding the layers helps clarify why this is not a trivial problem that will be fixed with the next model update.
The first layer is source quality. Even when the citation is real and the link works, the source is often a health media site or a patient-facing summary rather than the primary research. The AI gives you a real URL that leads to a real page, but that page is someone else's interpretation of the study — not the study itself. You are getting secondhand evidence presented as if it were firsthand.
The second layer is source relevance. The cited page may discuss the topic you asked about but not actually contain the specific claim the AI made. The AI states that "the 5-year survival rate is 73%," and cites a hospital blog post. You click through and the blog post discusses survival in general terms but never mentions the specific number. The AI interpolated between the source and the claim, and the citation provides a veneer of support without actual substance.
The third layer — and the most dangerous — is citation hallucination. The AI generates a reference that looks real but does not exist. Author names that sound plausible. A journal title that exists. A volume and page number that looks right. But when you search for it, there is no such article. The AI fabricated the citation the same way it fabricates text: by pattern matching on what academic references look like, not by looking up what actually exists.
Why hallucinated citations are uniquely dangerous in medicine
In most fields, a hallucinated citation is embarrassing. In medicine, it is dangerous. Consider what happens when a clinician uses an AI tool to look up a treatment recommendation and the AI returns a confident answer with a fabricated citation.
The clinician sees the citation and assumes the recommendation is evidence-based. They may not have time to verify it — they are in clinic, they are between cases, they are on call. They make a decision based on the AI's answer. The recommendation was wrong because the "evidence" behind it never existed. But it looked real. It had all the formatting of a real citation. It even had a plausible-sounding author and journal.
The pernicious thing about hallucinated citations is that they exploit exactly the trust mechanism that evidence-based medicine relies on. We are trained to trust claims that come with citations. We are trained to defer to peer-reviewed sources. When the AI generates a citation, it is hijacking that trust — using the form of evidence-based reasoning without any of its substance.
A confident AI answer without a citation is merely unhelpful. A confident AI answer with a hallucinated citation is actively misleading — it creates false confidence in a claim that may be wrong.
The verification burden falls on the clinician
Some will argue that clinicians should always verify AI-generated citations. This is true in principle and unrealistic in practice. The entire point of using an AI tool is to save time. If every answer requires the clinician to independently search for the cited reference, confirm it exists, access the full text, find the relevant passage, and verify that it supports the AI's claim — then the AI has not saved time. It has added a step.
The verification burden is particularly onerous for fabricated citations, because you cannot verify a non-existent source. You search PubMed for the cited article. Nothing comes up. You try Google Scholar. Nothing. You try the journal's website directly. Nothing. Now you have spent five minutes confirming that the citation does not exist, and you still do not have an answer to your original question. You have wasted time disproving the AI's fabrication.
For low-quality citations — real links to health media sites — the verification burden is different but equally time-consuming. You click the link, find a blog post, and realise it does not contain the specific claim the AI made. Now you need to find the actual primary source that the blog post was summarising, access that paper, and check the claim yourself. The AI has given you a detour, not a shortcut.
Why this is a structural problem, not a temporary one
There is a tempting belief that this will improve as AI models get better. Future models will be smarter, better calibrated, less prone to hallucination. This is partly true — newer models do hallucinate less frequently than older ones. But the structural problems remain.
General-purpose AI models will always search the open web, and the open web will always be dominated by SEO-optimised content rather than peer-reviewed research. As long as most medical literature is paywalled and most health media content is freely accessible, the citation quality problem is baked into the architecture. Better models will fabricate fewer citations, but they will still preferentially cite the same low-quality, freely available sources — because those are the sources they can access.
The hallucination problem may decrease in frequency but is unlikely to reach zero. Language models generate text probabilistically. There will always be edge cases where the model produces a plausible but non-existent reference. In casual use, a 2% hallucination rate is acceptable. In clinical decision-making, where each decision affects a patient, it is not.
The solution is not to wait for better models. The solution is to change the architecture — to give the AI access to the right sources in the first place, and to ensure that every citation points to a document you can verify.
The RAG alternative: ground the AI in your documents
Retrieval-Augmented Generation — RAG — solves the citation problem at the architectural level. Instead of allowing the AI to search the open web and cite whatever it finds, a RAG system constrains the AI to answer from a specific document collection — your documents, uploaded by you, verified by you.
When you ask a question in a RAG system, the answer comes from passages that were actually retrieved from your documents. The citation is not generated by the language model — it is attached to the retrieved passage. It points to a real document that exists in your collection, on a real page that you can open and read. The language model cannot hallucinate a citation because it is not producing citations — the retrieval system is.
This eliminates all three layers of citation failure. Source quality: the sources are your peer-reviewed papers, your clinical guidelines, your textbook chapters — not health blogs or commercial websites. Source relevance: the cited passage is the passage the AI used to generate the answer, so the connection between claim and evidence is direct, not inferred. Hallucination: impossible, because the citation comes from the retrieval system, not from the language model's text generation.
What this means for clinical practice
The practical implication is straightforward. If you are going to use AI to support clinical decisions, the AI must answer from the same evidence base you would consult yourself — your institution's clinical practice guidelines, the landmark trials in your specialty, the textbook chapters you trust, the systematic reviews you have vetted. Not from the open web. Not from health blogs. Not from sources selected by an algorithm optimised for internet search rather than clinical relevance.
This is not an abstract principle. It is a practical requirement. When you present a treatment recommendation at a multidisciplinary team meeting, it needs to be traceable to a guideline. When you discuss management options with a patient, the evidence supporting your recommendation should be identifiable. When a trainee asks you why you chose a particular approach, you should be able to point to the source.
AI that cites health blogs cannot meet this standard. AI that fabricates citations actively undermines it. AI that answers from your verified documents and cites the specific page in the specific paper is the only architecture that aligns with how evidence-based medicine actually works.
The question is not "can AI help with clinical questions?" It clearly can. The question is "does the AI answer from sources you would trust, and can you verify those sources in seconds?" If the answer is no, the tool is not ready for clinical use.
How Medevidex addresses this
Medevidex is built entirely on RAG architecture, specifically for medical documents. Every answer comes from passages retrieved from your uploaded literature. Every citation points to a specific document and page in your collection. Every source can be opened, read, and verified in the original PDF context.
There is no web search step. There is no mixing of web content with document content. The system answers from your documents or tells you it cannot find relevant information. This constraint is by design — it is the only way to guarantee that every citation is real, relevant, and verifiable.
The document processing pipeline is optimised for clinical content: multi-column guidelines, dense journal articles, textbook chapters with figures and tables. The embedding model is selected for medical vocabulary. The retrieval and ranking are tuned for the kinds of queries clinicians actually ask. The result is an AI tool that answers clinical questions the way evidence-based medicine demands — from the source, with the page number, traceable and verifiable.
This is not a feature. It is a fundamental requirement for any AI tool that claims to support clinical decision-making. If your citations are not real, your answers are not trustworthy. Everything else is secondary.
Read more
Why Medical AI Needs Real Citations · What Is RAG and Why Does It Matter for Clinical AI? · Medevidex vs ChatGPT, Consensus, and OpenEvidence