.Nature Medication, Released online: 26 July 2024 doi:10.1038/ s41591-024-03181-6Although sizable language models (LLMs) show promise in controlled setups, a study now subjects their limits in real-world medical applications and directs the method towards strong analysis and benchmarking prior to medical make use of.