Detection of the nipple in mammograms is an important step in algorithms for the detection of breast cancer. However, locating the nipple position is a challenging task due to distortion and displacement of the nipple by breast diseases, improper imaging techniques, and variation of the characteristics of breast tissues with different imaging protocols or modalities. This paper presents a novel approach for the detection of the nipple in mammograms based on the converging characteristics of oriented patterns of the breast tissues towards the nipple. The oriented structures are extracted with a bank of real Gabor filters and are transformed into the Radon domain to analyze linear structures of tissue patterns to determine the nipple position. The performance of the method was evaluated with different types of images, such as scanned screen-film (from the mini-MIAS and DDSM databases), digital radiography, and computed radiography, and average errors of 7.71 mm, 7.52 mm, 9.23 mm, and 12.10 mm were achieved, respectively, with reference to the nipple location marked by an expert radiologist. The proposed method outperforms two recently developed approaches for the same application.