Recent advances in targeted cancer therapy hold great promise for both research and clinical applications and push the boundaries in finding new treatments for various currently incurable cancers. However, these therapies require specific cell-targeting mechanisms for the efficient delivery of drug cargo across the cell membrane to reach intracellular targets and avoid diffusion to unwanted tissues. Traditional drug delivery systems suffer from a limited ability to travel across the barriers posed by cell membranes and, therefore, there is a need for high doses, which are associated with adverse reactions and safety concerns. Bacterial toxins have evolved naturally to specifically target cell subtypes via their receptor binding module, penetrating the cell membrane efficiently through the membrane translocation process and then successfully delivering the toxic cargo into the host cytosol. They have, thus, been harnessed for the delivery of various drugs. In this review, we focus on bacterial toxin translocation mechanisms and recent progress in the targeted delivery systems of cancer therapy drugs that have been inspired by the receptor binding and membrane translocation processes of the anthrax toxin protective antigen, diphtheria toxin, and Pseudomonas exotoxin A. We also discuss the challenges and limitations of these studies that should be addressed before bacterial toxin-based drug delivery systems can become a viable new generation of drug delivery approaches in clinical translation.