The detection of Mycobacterium tuberculosis complex (MTB) in clinical specimens is important for diagnosing and caring for patients in whom tuberculosis is clinically suspected. We collected 129 FFPE specimens, including 56 nontuberculosis cases, 63 MTB cases, and 10 nontuberculous mycobacteria (NTM) cases determined by acid-fast bacilli (AFB) culture. We performed AFB staining; nested MTB PCR, targeting the IS6110 gene; and real-time MTB PCR, targeting the senX3-regX3 intergenic region in the 129 FFPE specimens. The sensitivity and specificity of AFB staining were 37.0% and 98.2%, respectively, using AFB culture results as the reference standard. The sensitivity and specificity of detecting MTB were 68.3% and 98.5%, respectively, by nested PCR; and 74.6% and 98.5% by real-time PCR, respectively. Among the 129 specimens, four were positive by AFB staining but negative by nested or real-time PCR. NTM grew in all four of these cases by AFB culture. AFB density in FFPE tissue sections significantly correlated with MTB DNA load. Thus, real-time PCR is a useful diagnostic tool for rapid and sensitive MTB detection in FFPE specimens, whereas NTM should be included in differential diagnoses of cases positive by AFB staining but negative by PCR.