Tumor cells exposed to a physiological matrix of type I collagen fibers form elongated collagenolytic invadopodia, which differ from dotty-like invadopodia forming on the gelatin substratum model. The related scaffold proteins, TKS5 and TKS4, are key components of the mechanism of invadopodia assembly. The molecular events through which TKS proteins direct collagenolytic invadopodia formation are poorly defined. Using coimmunoprecipitation experiments, identification of bound proteins by mass spectrometry, and in vitro pull-down experiments, we found an interaction between TKS5 and FGD1, a guanine nucleotide exchange factor for the Rho-GTPase CDC42, which is known for its role in the assembly of invadopodial actin core structure. A novel cell polarity network is uncovered comprising TKS5, FGD1, and CDC42, directing invadopodia formation and the polarization of MT1-MMP recycling compartments, required for invadopodia activity and invasion in a 3D collagen matrix. Additionally, our data unveil distinct signaling pathways involved in collagenolytic invadopodia formation downstream of TKS4 or TKS5 in breast cancer cells.