Artificial intelligence (AI) is being characterized as a medical device that requires guidance to ensure its safe use in health care. The authors highlight existing authority through the Centers for Medicare & Medicaid Services (CMS) and other government entities to track and respond to instances of AI use that result in patient harm. They also call for specific training developments that enable clinicians to apply AI-generated information effectively in front-line care.