How can I fact-check the information that ChatGPT and other large language models (LLMs) give me?
Answer
Because LLMs generate their output, their claims aren't reliable. Basically, you need to find other sources to make sure what they've claimed is true. We call this process lateral reading [Website]. Start by identifying specific claims in the text, like, "the US dropped three atomic bombs on Japan in WWII." Some LLMs will include a link to a website they think they've scraped the data from. Follow the link to make sure the website exists, and if it does, does it say what the LLM reported? If the answer to either of those is no, the claim is probably a hallucination. Even if the both answers are yes, or if it didn't cite its source, copy the claim and paste it into Google. Make sure to scroll past the AI summary and the sponsored posts so you can evaluate the real search results. Do those websites say the same thing? What does WIkipedia say? All of these should tell you that the claim is false, the US dropped only two atomic bombs.
Similarly, if the LLM gives you a citation, you'll want to make sure the paper is real and that it applies to the topic. Copy and paste the citation into Google Scholar [Website] or our library's OneSearch [Website]. Do you get any results? If so, check the paper's abstract to make sure it's actually about what you've asked.
While this can seem like extra work, you'll get faster at it the more you practice. And it's a very useful habit for vetting information, in school and out of it. Learn more about lateral reading in the following video.
Image courtesy of the University Libraries and the Teaching and Learning Transformation Center (TLTC) at the University of Maryland via the AI and Information Literacy Canvas Course, CC BY-NC 4.0.