Medical large language models are vulnerable to data-poisoning attacks.

To produce safe, accurate output, large language models (LLMs) must be trained on accurate information. In this study, researchers simulated a data-poisoning attack by implanting false medical information into a popular LLM training dataset. Results show that even a small amount of medical misinformation in the training dataset can result in harmful models that could compromise patient safety.