In most of the world’s countries, scholarship evaluation for tenure and promotion continues to rely on conventional criteria of publications in journals of high impact factor and achievements in securing research funds. Continuing to hire and promote scholars based on these criteria exposes universities to risk because students, directly and indirectly through government funds, are the main source of revenues for academic institutions. At the same time, talented young researchers increasingly look for professors renowned for excellence in mentoring doctoral students and early career researchers. Purposeful scholarship evaluation in the open science era needs to include all three areas of scholarly activity: research, teaching and mentoring, and service to society.