Clinical decision-making is a complex process requiring consideration of information from multiple sources. This systematic review, including 79 studies, explored areas for improvement with medical question answering (QA) systems, which are designed to answer clinicians’ questions in real time based on current evidence. The authors identified several characteristics that could be used to improve future QA systems, such as providing more robust answers or the use of more complex datasets for system development.