LLM LLMitations

---

Bolus Dose EP. 04

In this episode, we discuss some concerns about LLM implementation in the healthcare setting, including the following topics:

• More detail about hallucinations, issues with accuracy 

• Difficulties of model evaluation

• Concerns regarding bias and equity

• Governance and monitoring of LLMs in a healthcare settings

Some resources and papers we discuss: 

Li H, Moon JT, Iyer D, Balthazar P, Krupinski EA, Bercu ZL, Newsome JM, Banerjee I, Gichoya JW, Trivedi HM. Decoding radiology reports: Potential application of OpenAI ChatGPT to enhance patient understanding of diagnostic reports. Clin Imaging. 2023 Sep;101:137-141. 

Examples of shift in AI models including ChatGPT:

• Koch LM, Baumgartner CF, Berens P. Distribution shift detection for the postmarket surveillance of medical AI algorithms: a retrospective simulation study. NPJ Digit Med. 2024 May 9;7(1):120.  

• Chen, Lingjiao et al. “How is ChatGPT's behavior changing over time?” ArXiv abs/2307.09009 (2023)

Discussion of LLMs perpetuating bias:

• Omiye, J.A., Lester, J.C., Spichak, S. et al. Large language models propagate race-based medicine. npj Digit. Med. 6, 195 (2023).

• Zack T, Lehman E, Suzgun M, Rodriguez JA, Celi LA, Gichoya J, Jurafsky D, Szolovits P, Bates DW, Abdulnour RE, Butte AJ, Alsentzer E. Assessing the potential of GPT-4 to perpetuate racial and gender biases in health care: a model evaluation study. Lancet Digit Health. 2024 Jan;6(1):e12-e22. doi: 10.1016/S2589-7500(23)00225-X.