Background:Artificial Intelligence (AI)-based automatic lung nodule detection system improves the detection rate of nodules. It is important to evaluate the clinical value of the AI system by comparing AI-assisted nodule detection with actual radiology reports.Objective:To compare the detection rate of lung nodules between the actual radiology reports and AI-assisted reading in lung cancer CT screening.Methods:Participants in chest CT screening from November to December 2019 were retrospectively included. In the real-world radiologist observation, 14 residents and 15 radiologists participated in finalizing radiology reports. In AI-assisted reading, one resident and one radiologist reevaluated all subjects with the assistance of an AI system to locate and measure the detected lung nodules. A reading panel determined the type and number of detected lung nodules between these two methods.Results:In 860 participants (57±7 years), the reading panel confirmed 250 patients with >1 solid nodule, while radiologists observed 131, lower than 247 by AI-assisted reading (p<0.001). The panel confirmed 111 patients with >1 non-solid nodule, whereas radiologist observation identified 28, lower than 110 by AI-assisted reading (p<0.001). The accuracy and sensitivity of radiologist observation for solid nodules were 86.2% and 52.4%, lower than 99.1% and 98.8% by AI-assisted reading, respectively. These metrics were 90.4% and 25.2% for non-solid nodules, lower than 98.8% and 99.1% by AI-assisted reading, respectively.Conclusion:Comparing with the actual radiology reports, AI-assisted reading greatly improves the accuracy and sensitivity of nodule detection in chest CT, which benefits lung nodule detection, especially for non-solid nodules.