The usefulness of current hospital quality ratings has been called into question. This commentary outlines weaknesses in the Centers for Medicare and Medicaid Services (CMS) star rating system. The authors offer recommendations to CMS to improve their comparison program, including reducing the number of measures tracked, investing more in quality measurement at a national level, and developing a "rate-the-ratings" system.