Artificial intelligence and machine learning are often promoted as tools with impressive potential for generating rapid improvements in healthcare decision making. This news story highlights how underlying problems and testing failures limit the trustworthiness of conclusions drawn by the systems.