Large language models (LLM) offer a promising approach to improving diagnostic accuracy. In this study, internal medicine physicians were randomized to use conventional (eg, UpToDate) or conventional plus LLM diagnostic resources to provide a differential and final diagnosis on 4 to 6 clinical vignettes. There was no significant difference in diagnostic performance or time spent per case between conventional and conventional plus LLM groups; LLM alone performed 16% better than the control group.