Student evaluation is a key consideration for educational program administrators because program success depends on students’ ability to demonstrate successful development of core competencies. Student evaluations must therefore be aligned with learning objectives and overall program goals. Graduate level educational programs typically incorporate course-level and program-level evaluations, e.g., a final examination in a single course vs. a qualifying examination that assesses knowledge gained from several courses. While there is often considerable attention given to the structure of these evaluations at the program level, the format is typically left to the instructor’s discretion. We argue in this article that there are administrative advantages to encouraging instructors to adopt creative forms of assessment that extend beyond the typical concerns related to program structure. Specifically, we argue that advantages can be gained in terms of increasing student engagement, adding real world context to student evaluations, maintaining positive program culture, and reducing the opportunity for cheating. We present two examples of creative assessments implemented in a 2-year Master of Biostatistics program, along with a discussion of three key questions administrators should consider as they work with instructors to develop innovative assessment methods: (1) what changes to make; (2) in what order to make those changes; and (3) how to consult with instructors about making those changes.