Patients are increasingly using the internet and conversational agents (CAs) like Siri, Alexa, and Google to find answers to their healthcare questions. Investigators used these CAs to detect errors and failures in calculating correct insulin doses. Failure classes include misunderstanding and misrecognition of words. Potential failures must be considered before deployment of CAs in safety-critical environments.