Medical imaging, including X-ray, computed tomography (CT), and magnetic resonance imaging (MRI), plays a critical role in early detection, diagnosis, and treatment response prediction of cancer. To ease radiologists’ task and help with challenging cases, computer-aided diagnosis has been developing rapidly in the past decade, pioneered by radiomics early on, and more recently, driven by deep learning. In this mini-review, I use breast cancer as an example and review how medical imaging and its quantitative modeling, including radiomics and deep learning, have improved the early detection and treatment response prediction of breast cancer. I also outline what radiomics and deep learning share in common and how they differ in terms of modeling procedure, sample size requirement, and computational implementation. Finally, I discuss the challenges and efforts entailed to integrate deep learning models and software in clinical practice.