Resident preparedness to provide safe care without supervision has been found to be inconsistent across health care. This commentary explains how written tests can fail to uncover these gaps in competency and suggests that current checklists for basic procedural skills are insufficient for assessing performance. The authors recommend an itemized assessment tool with a sharper focus on competency and review their experience with implementing one.