Breast cancer is a lethal illness that has a high mortality rate. In treatment, the accuracy of diagnosis is crucial. Machine learning and deep learning may be beneficial to doctors. The proposed backbone network is critical for the present performance of CNN-based detectors. Integrating dilated convolution, ResNet, and Alexnet increases detection performance. The composite dilated backbone network (CDBN) is an innovative method for integrating many identical backbones into a single robust backbone. Hence, CDBN uses the lead backbone feature maps to identify objects. It feeds high-level output features from previous backbones into the next backbone in a stepwise way. We show that most contemporary detectors can easily include CDBN to improve performance achieved mAP improvements ranging from 1.5 to 3.0 percent on the breast cancer histopathological image classification (BreakHis) dataset. Experiments have also shown that instance segmentation may be improved. In the BreakHis dataset, CDBN enhances the baseline detector cascade mask R-CNN (mAP = 53.3). The proposed CDBN detector does not need pretraining. It creates high-level traits by combining low-level elements. This network is made up of several identical backbones that are linked together. The composite dilated backbone considers the linked backbones CDBN.