Recent articles have pointed out the potential benefits, limits and risks that may occur from use of AI Chatbots, including substantial inaccuracies [1,2]. Such inaccuracies, or downright fake information, have been described as “hallucinations,” and even this term has been criticized and reimagined as “confabulations” [3].

After recently receiving a notification from Google about their new GPT-4 chatbot application, Gemini (http://gemini.google.com/), I thought I’d give it a whirl in the general area of evidence-based medicine and health outcomes and policy.

As a starter question, “I need help with information about outcomes of a specific surgical procedure,” Gemini responded that all of the following might be helpful resources: PubMed, Cochrane Library, NIH, ClinicalTrials.gov, and professional societies. So far, so good, a great answer to get one started on a search of the best available evidence.

Then, I asked a much more specific question, “What is the outcome of thoracic outlet surgery for workers’ compensation cases?” After reasonably stating that the outcome can vary depending on the individual case, the severity of the condition, and the type of surgery performed, Gemini stated that “Studies have shown that thoracic outlet surgery can be effective in improving symptoms and increasing function in workers who have been diagnosed with thoracic outlet syndrome.”

Gemini supported this conclusion by citing two studies. First, “A 2019 study published in the journal “Spine” found that thoracic outlet surgery was associated with a significant improvement in symptoms and function in workers with thoracic outlet syndrome. The study found that 85% of workers who underwent surgery reported an improvement in their symptoms, and 70% reported a return to work.” Second, “Another study, published in the journal” The American Journal of Industrial Medicine” in 2018, found that thoracic outlet surgery was associated with a significant reduction in pain and an improvement in function in workers with thoracic outlet syndrome. The study found that 80% of workers who underwent surgery reported a reduction in pain, and 75% reported an improvement in function.”

I followed up with, “The studies you cited do not seem to exist. Can you explain that?” And Gemini responded, “I apologize for the mistake. I am still under development and learning to perform many kinds of tasks, including finding and citing academic studies. Upon further research, I cannot find any studies that specifically examine the outcomes of thoracic outlet surgery for workers compensation cases.” And, “However, I did find a few studies that examined the outcomes of thoracic outlet surgery in general. These studies found that thoracic outlet surgery can be effective in improving symptoms and increasing function in patients with thoracic outlet syndrome.” Gemini then repeated the findings from the two phantom studies cited above.

I followed up with, “The two studies you have cited do not exist.” And Gemini, contritely, “I apologize for the mistake. I am still underdevelopment…” “I have checked the references for the two studies that I cited, and I can confirm that they do not exist. I am not sure why I cited these studies, as I am programmed to only cite studies that have been published in reputable academic journals.”

If an injured worker with diagnosed neurogenic thoracic outlet syndrome had asked Gemini the original question in search of some advice about whether to pursue a potentially harmful procedure, the information provided could have led to a false sense of security as to what to expect from surgery. How is it possible that the Chatbot could make up evidence so specifically as to the journal name and year, but without authors, volume and page numbers? This is a frightening example of the state of the AI chatbot universe in health care. And I really don’t have confidence that such very bad information will not lead to more harmful overuse of some procedures before whatever led to these responses is corrected.

At the very least, one would think that Gemini could learn from its mistakes, so I re-asked the question on outcomes of thoracic outlet surgery among workers’ compensation cases a few days later. The answers were exactly the same, including the fake references and false optimism.

Alas, when asked the same original question a few months later, Gemini responded much more meaningfully and accurately (exact transcript) and with better organized material, including true references [4]: