Understanding the structural origins of cationic diffusion processes in silicate glasses is important for high-tech applications of silicate glasses. For glasses with more than one network former, transport properties such as diffusivity are often nonlinear functions of the particular distribution of these network formers, a phenomenon known as the mixed network former effect. Here, we investigate the sodium-potassium interdiffusion (DNa-K) and the calcium inward diffusion (DCa) in soda lime borosilicate glasses with varying silica/borate ratio but constant modifier content. Indeed, the structural organization of borosilicate glasses results in a pronounced nonlinear composition dependence of DNa-K and DCa (i.e., the mixed network former effect). Initial addition of B2O3 to the glass system results in a significant decrease in both diffusivities, whereas the change in diffusivity per mole of added B2O3 decreases with increasing B2O3 concentration. Besides the influences of water content and atomic packing degree, we find that 99% of the composition dependence of log DNa-K can be ascribed to the change in concentration of tetrahedral boron groups. This indicates that the formation of BO4/2 groups slows down diffusion processes of alkali and alkaline earth ions. Therefore, the mixed network former effect of the studied glass series is linked with the change of the concentration of tetrahedral boron groups, which is caused by the interactions between the different types of network formers.