Abstract Objectives To compare the exposure data generated by using the Canadian job-exposure matrix (CANJEM) with data generated by expert assessment, for jobs held by women. Methods We selected 69 occupational agents that had been assessed by experts for each of 3403 jobs held by 998 women in a population-based case–control study of lung cancer. We then assessed the same agents among the same jobs by linking their occupation codes to CANJEM and thereby derived probability of exposure to each of the agents in each job. To create binary exposure variables, we dichotomized probability of exposure using two cutpoints: 25 and 50% (referred to as CANJEM-25% and CANJEM-50%). Using jobs as units of observation, we estimated the prevalence of exposure to each selected agent using CANJEM-25% and CANJEM-50%, and using expert assessment. Further, using expert assessment as the gold standard, for each agent, we estimated CANJEM’s sensitivity, specificity, and kappa. Results CANJEM-based prevalence estimates correlated well with the prevalences assessed by the experts. When comparing CANJEM-based exposure estimates with expert-based exposure estimates, sensitivity, specificity, and kappa varied greatly among agents, and between CANJEM-25% and CANJEM-50% probability of exposure. With CANJEM-25%, the median sensitivity, specificity, and kappa values were 0.49, 0.99, and 0.46, respectively. Analogously, with CANJEM-50%, the corresponding values were 0.26, 1.00, and 0.35, respectively. For the following agents, we observed high concordance between CANJEM- and expert-based assessments (sensitivity ≥0.70 and specificity ≥0.99): fabric dust, cotton dust, synthetic fibres, cooking fumes, soldering fumes, calcium carbonate, and tin compounds. We present concordance estimates for each of 69 agents. Conclusions Concordance between CANJEM and expert assessment varied greatly by agents. Our results indicate which agents provide data that mimic best those obtained with expert assessment.