As more hospitals begin to implement computerized provider order entry (CPOE) systems, rigorously evaluating their real-world performance at preventing medication errors has become crucial. The Leapfrog Group was an early pioneer in calling for wider CPOE implementation, and this study reports on the validation of a tool developed by Leapfrog for assessing the ability of CPOE systems to prevent serious errors. The tool, which uses simulated cases, proved to be effective, as the incidence of errors it detected corresponded closely to the actual error rates of participating hospitals. Prior simulation research has shown that many commercial systems fail to detect even potentially serious errors, and this study provides reassurance that CPOE systems that pass the Leapfrog evaluation are likely to successfully prevent medication errors.