Opening the book…
Accuracy and Fabricated Information
The Mystery: How Can a Citation Look Perfect and Still Not Exist?
Elena asks a generative AI tool to recommend scholarly sources about food insecurity among community college students. The tool returns a polished list containing authors, article titles, journal names, volume and issue numbers, page ranges, and DOIs.
One citation seems perfect. Its title matches Elena’s topic exactly, and the journal sounds reputable. She copies it into her paper and searches the college library for the full article.
Nothing appears.
She searches Google Scholar. Still nothing. She enters the DOI into a DOI lookup service, but it leads nowhere. When she asks the AI whether the citation is real, the tool apologizes and produces a replacement.
The replacement also does not exist.
How can a system generate a citation containing all the parts of a scholarly article without describing any actual publication?
To solve the mystery, Elena must understand that fluent language and factual accuracy are related—but they are not the same thing.
Generative AI Can Be Useful and Wrong
Generative AI can produce accurate explanations, useful summaries, strong examples, and helpful starting points. It can also produce information that is partly wrong, completely invented, outdated, oversimplified, or unsupported by the cited source.
The difficulty is that accurate and inaccurate responses may sound equally confident.
A language model does not automatically pause, look worried, and announce, “I am about to invent a journal article.” It may present false information using the same polished grammar, academic vocabulary, and organized formatting it uses for correct information.
This is why an AI response should not be treated as evidence merely because it sounds knowledgeable.
Accuracy must be checked outside the response.
Accuracy refers to how closely information matches reliable evidence or established facts.
Fabricated information is content that appears factual but was invented rather than supported by a real source or event.
An AI hallucination is the common term for generated content that is unsupported, inconsistent with the available evidence, or factually incorrect.
A fabricated citation names a source that does not actually exist.
A citation error occurs when a real source is described incorrectly, such as through a wrong author, title, date, journal, page range, or DOI.
Grounding means connecting an AI response to identifiable evidence, such as a retrieved document, webpage, dataset, or source supplied by the user.
The term hallucination is widely used in AI research, but it is a metaphor. The system is not experiencing a human sensory hallucination. It is generating unsupported or incorrect content.
Why Fabrication Happens
A generative AI model produces language by estimating what text is likely to come next. That process can create a response that matches the usual form of an answer without matching the underlying facts.
For example, scholarly citations follow recognizable patterns. They usually contain author names, publication years, article titles, journal titles, volume numbers, pages, and DOIs. A model can reproduce that pattern even when it does not have reliable evidence for a specific article.
It may combine the name of one real researcher, the topic of another article, the title pattern of a familiar journal, and a DOI-shaped string into one convincing but fictional reference.
The result looks like scholarship because it imitates the structure of scholarship.
It is not necessarily scholarship.
Fabrication Is Not the Same as Deliberate Lying
A person lies when they know a statement is false and intentionally present it as true.
A language model does not possess human intentions or beliefs in the ordinary sense. It generates a response from learned patterns, instructions, retrieved information, and the conversation context.
Calling the model a liar can therefore hide the real problem. The system may produce false information without recognizing the conflict between the response and the evidence.
For the student, the practical result remains serious: incorrect information can enter an assignment, decision, or conversation.
The absence of human intent does not make the output accurate.
Common Accuracy Problems
AI errors do not all look the same. Some are easy to catch because they describe an obviously impossible event. Others change only one name, date, statistic, or qualification.
| Type of problem | Example | Why it is risky |
|---|---|---|
| Completely fabricated information | The AI invents a court case, study, quotation, or historical event | The student may cite evidence that never existed |
| Fabricated citation | The authors, title, journal, and DOI form a realistic-looking but fictional reference | Detailed formatting can create false confidence |
| Misrepresented source | A real article exists, but it does not support the AI’s claim | The citation appears to verify a statement that the source never made |
| Blended information | Details from several real people, studies, or events are combined into one account | Much of the response may sound familiar, making the invented combination difficult to notice |
| Wrong detail | The explanation contains an incorrect date, statistic, name, or location | A small error can change the meaning of an otherwise accurate account |
| Missing qualification | A correlation becomes causation, or a finding from one sample becomes a universal conclusion | The response exaggerates what the evidence supports |
| Outdated information | The answer reflects an earlier law, officeholder, product version, or policy | The information may once have been accurate but no longer applies |
Figure 8.3.1 A response can contain more than one of these problems at the same time. (CC BY-NC; Cheryl Bailey)
Real Citations Can Still Be Used Incorrectly
Finding that a cited article exists does not finish the evaluation.
An AI tool may cite a real study but describe its findings inaccurately. It may claim that researchers proved causation when they reported only an association. It may apply findings from laboratory mice to humans, from adolescents to all adults, or from one country to the entire world.
It may also cite a source that mentions the general topic without supporting the exact sentence.
Suppose an AI response states:
A 2022 study proved that food pantries increase college graduation rates.
The cited article may actually be a survey showing that students valued a pantry. The article is real. The claim is still unsupported.
Verification therefore requires asking two questions:
Does the source exist?
Does the source actually support this claim?
Students need both answers.
A Real Study of Fabricated Citations
Walters and Wilder (2023) tested citations generated by GPT-3.5 and GPT-4 in response to prompts requesting scholarly literature. In their sample, 55% of the GPT-3.5 citations and 18% of the GPT-4 citations were fabricated. Among the citations that referred to real sources, substantial portions still contained errors. The researchers concluded that improved models reduced—but did not eliminate—fabrication and bibliographic mistakes.
Those percentages should not be treated as permanent accuracy rates for every AI tool. The study examined particular model versions, prompts, subjects, and conditions in 2023. AI systems change rapidly, and performance varies by task.
The broader lesson remains useful: a citation can look complete and still require verification.
There Is No Single AI Accuracy Score
Students sometimes ask, “How accurate is AI?”
There is no one percentage that answers that question.
Accuracy depends on the model, version, prompt, topic, language, available context, retrieval system, and type of task. A tool may perform well when explaining a common concept and poorly when asked for an obscure citation. It may accurately summarize a supplied article but invent details when asked to answer from memory.
Performance can also change after an update.
A score from one benchmark does not guarantee that the same tool will be accurate for a different question. A model that performs well on a graduate examination may still produce a fabricated reference, inaccurate local policy, or incorrect current fact. Research has documented both strong performance on some academic tasks and serious failures in specialized applications.
The correct question is not simply, “Is this AI accurate?”
Ask:
Accurate at what task, under what conditions, and compared with what evidence?
Confidence Is a Writing Style, Not an Accuracy Measurement
AI-generated language often sounds certain because direct, confident sentences are common in explanatory writing.
A sentence such as “Researchers have conclusively demonstrated…” may reflect strong evidence. It may also be an unsupported phrase selected because it fits the expected style.
The tone does not reveal whether the system checked a source.
Likewise, cautious wording does not automatically make a response accurate. A fabricated statement can include may, suggests, and possibly.
Students should treat confidence as a feature of the writing, not proof of the information.
Asking the AI to Check Itself Is Not Independent Verification
When Elena cannot find the fictional article, she asks the same AI tool whether the citation is correct.
This is better than noticing nothing, but it is not independent verification.
The tool may repeat the original answer, change part of it, apologize without identifying the error, or replace one false citation with another. A second response is still produced by the same type of system.
The model may sometimes identify and repair its own mistakes. It may also sound highly reflective while remaining wrong.
A reliable check moves outside the generated answer. Elena should search the library database, publisher website, Google Scholar, or another appropriate source.
The AI can assist with verification. It should not be the only verifier of its own output.
Different Answers Do Not Automatically Reveal Which One Is Correct
Elena repeats the same prompt and receives a different response.
This variation is possible because generative systems may choose among several likely language sequences. Model settings, system updates, conversation context, and retrieved sources can also change the output.
A changed answer does not prove that the first answer was false or that the second is correct.
Likewise, receiving the same answer twice does not independently confirm it. The system may repeat the same unsupported pattern.
Consistency can be useful, but it is not evidence by itself.
AI May Continue an Error Introduced by the User
Suppose a student asks:
Why did the Supreme Court decide Garcia v. California in 2024?
If no such decision exists, a model may correct the premise. It may also accept the fictional case and generate an explanation.
Research has shown that models can elaborate on fabricated information placed inside prompts, including in high-stakes clinical scenarios. This means a confident question can sometimes pull the model deeper into an error rather than prompting it to challenge the assumption.
Students should therefore verify the premise of a question, not only the answer.
A polished explanation of a nonexistent event remains a polished explanation of a nonexistent event.
Summaries Can Introduce New Claims
AI tools are often useful for summarizing long text, but the output should still be compared with the original.
A summary may omit an important exception, combine findings from separate sections, change tentative language into a definite conclusion, or add background information not found in the document.
Suppose an article states:
Within this sample, students who used the service reported somewhat higher confidence, although the study did not measure academic performance.
An inaccurate summary might become:
The service improves students’ grades and confidence.
The sentence is shorter. It is also stronger than the evidence.
When accuracy matters, the student should check names, numbers, findings, limitations, and quotations against the original source.
Quotations Require Special Caution
A generated quotation should be considered unverified until it can be found in the original source.
AI may reproduce a real quotation accurately. It may paraphrase an idea and place quotation marks around it. It may blend several statements. It may invent a quotation that sounds appropriate for the person.
This can be especially difficult to detect with famous people because the invented line may resemble something they might have said.
Never cite a quotation merely because an AI tool attributed it to someone. Locate the speech, interview, book, court decision, transcript, or other original record.
A quotation needs a source location, not just a famous name standing nearby.
Numbers Can Look Precise While Being Wrong
Specific numbers create credibility.
An AI response may state that “63.7% of students” experienced an outcome, even when no source supports that exact figure. It may also report a real statistic from the wrong year, population, or study.
Precision should increase scrutiny, not reduce it.
When an AI provides a statistic, ask where the number came from, what population it describes, when the data were collected, how the variable was measured, and whether the source can be opened.
A decimal point is not a tiny laboratory coat.
Retrieval Can Improve Accuracy but Does Not Guarantee It
AI search systems and retrieval-augmented tools may connect the model to current webpages, uploaded documents, or scholarly collections.
This can reduce some forms of fabrication because the response has evidence available during generation. It can also make sources easier to inspect.
However, retrieval does not automatically guarantee that the generated sentence matches the source. The system may select an irrelevant passage, overlook contradictory evidence, misunderstand a table, or combine several documents incorrectly.
A citation-enabled answer is generally easier to verify than an answer with no sources. It is not self-verifying.
The user still needs to open the cited material.
Accuracy Depends on the Stakes
Not every AI mistake creates the same harm.
An invented detail in a brainstorming exercise may be annoying. An invented medication instruction, legal deadline, financial requirement, or criminal accusation can cause serious harm.
The higher the stakes, the stronger the verification should be.
For college research, fabrication can lead to unsupported claims, false citations, inaccurate quotations, and academic-integrity problems. In health, law, finance, safety, or public policy, the same behavior can affect real decisions.
The proper response is not to panic whenever AI generates a sentence. It is to match the checking process to the consequences of being wrong.
Expert Thinking: Elena Verifies the “Perfect” Citation
Elena stops asking whether the citation looks scholarly.
She searches the exact article title in quotation marks. No results appear.
She searches the authors’ names and topic. The authors exist, but they did not write the article.
She searches the journal’s archive for the stated year, volume, and issue. The page range belongs to a different article.
She checks the DOI. It is not registered.
Elena now has several independent reasons to conclude that the citation was fabricated.
She removes it from her paper and searches the library database using the useful keywords from the fictional title. The false citation was not evidence, but its wording helped her develop a better search.
Her research note becomes:
The AI-generated citation appeared complete but could not be verified through the journal, author records, databases, or DOI system. It should not be cited. Its keywords may still be used to search for real scholarship.
Elena salvages the useful part without preserving the false part.
Course Connection: Fluency Is Not Authority
College research requires evidence that readers can locate and evaluate.
Generative AI can help explain terminology, generate search language, compare verified notes, or suggest possible directions. It cannot make an unsupported claim reliable by expressing it clearly.
The researcher remains responsible for determining whether a source exists, whether a quotation is authentic, whether a statistic is accurate, and whether the cited evidence supports the conclusion.
This responsibility does not disappear because the error originated with software.
Solving the Mystery
The citation looked perfect because the AI system generated a highly recognizable citation pattern.
It combined realistic authors, a plausible article title, a scholarly-sounding journal, publication details, and a DOI-shaped string. The parts fit the expected format, but they did not point to a real publication.
The model had produced the appearance of bibliographic evidence without the evidence itself.
Elena revises her approach:
An AI-generated citation is a lead, not a verified source. Before using it, I must locate the publication through an independent research system, confirm its bibliographic details, read the source, and verify that it supports my claim.
The mystery was not how a fake citation could look so strange.
It was how a fake citation could look so normal.
That normal appearance is exactly why verification matters.
Check Your Understanding
Why can an AI-generated answer sound confident even when it is incorrect? What is the difference between a fabricated citation and an inaccurate citation to a real source? Why is asking the same AI tool to check itself not independent verification? How could a real article be cited in support of a claim it does not actually make? Why should a precise statistic receive more scrutiny rather than less? What steps would you take before including an AI-generated quotation in an assignment?
Generative AI can produce accurate information, inaccurate information, and mixtures of both within the same response.
The term hallucination commonly describes generated content that is false, unsupported, or inconsistent with the available evidence. The term does not mean that the system experiences human perception or intentionally lies.
Fluent wording, confident tone, detailed formatting, and specific numbers do not prove accuracy.
AI tools can fabricate sources, quotations, statistics, events, names, and supporting details. They can also misrepresent real sources.
No single accuracy rate applies to every AI system or task. Performance changes across models, versions, topics, prompts, and research conditions.
Retrieval and citations can improve grounding, but students must still open and evaluate the sources.
Researchers should independently verify important claims and remove information that cannot be supported.
References
Huang, Lei, et al. "A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questions." ACM Transactions on Information Systems, vol. 43, no. 2, 2025, article 42.
Omar, Mahmud, et al. "Multi-Model Assurance Analysis Showing Large Language Model Vulnerability to Adversarial Hallucination Attacks in Clinical Prompts." Communications Medicine, vol. 5, 2025.
Soroush, Ali, et al. "Large Language Models Are Poor Medical Coders—Benchmarking of Medical Code Querying." NEJM AI, vol. 1, no. 5, 2024.
Stribling, Daniel, et al. "The Model Student: GPT-4 Performance on Graduate Biomedical Science Exams." Scientific Reports, vol. 14, 2024, article 5670.
Walters, William H., and Esther Isabel Wilder. "Fabrication and Errors in the Bibliographic Citations Generated by ChatGPT." Scientific Reports, vol. 13, 2023, article 14045.
Attributions
No third-party AI responses, fabricated citations, screenshots, articles, or database records were reproduced or adapted. Elena’s mystery, sample claims, verification process, comparison table, and knowledge-check questions constitute original instructional content created for the LIB 15 OER.
The real-world citation case draws on Walters and Wilder’s study of fabricated and inaccurate references generated by GPT-3.5 and GPT-4. The broader explanation of hallucination draws on Huang et al. Examples involving strong performance in one context and poor reliability in another draw on Stribling et al. and Soroush et al.
This page titled 8.2: Accuracy and Fabricated Information was last modified on Mon, 05 Oct 2026 16:52:09 GMT and is shared under a CC BY-NC-SA 4.0 license and was authored, remixed, and/or curated by Cheryl Bailey.
The book could not be loaded just now.