Numerous instruments have been developed to measure gaming-related health problems based on “internet gaming disorder” (IGD) in the third section of the Diagnostic and Statistical Manual of Mental Disorders (5th ed.) and “gaming disorder” (GD) in the International Classification of Diseases (11th rev.). However, the criteria in the manuals tend to be operationalized in numerous diverse ways, which can make screening outcomes incomparable. A content validity analysis is needed to reassess the relationships between the diagnostic criteria and the items that operationalize them. The IGD and GD criteria were divided into sematic components. A qualitative content validity analysis was carried out for all items employed by the 17 instruments that claim to measure either construct by their criteria in English. In all but one instrument, the operationalizations did not include all criterion components. There were two main reasons found for this: the components had simply been left out or had been alternatively modified into other components. Criteria that were vaguely described in the manuals were sources of lower content validity items. The study implies that many of the problems in IGD and GD measurement derive from criteria operationalization and original manual descriptions. The conclusion provides practical recommendations that researchers can apply to improve the content validity of their measurement.