Interstitial atoms usually diffuse much faster than vacancies, which is often the root cause for the ineffective recombination of point defects in metals under irradiation. Here, via ab initio modeling of single-defect diffusion behavior in the equiatomic NiCoCrFe(Pd) alloy, we demonstrate an alloy design strategy that can reduce the diffusivity difference between the two types of point defects. The two diffusivities become almost equal after substituting the NiCoCrFe base alloy with Pd. The underlying mechanism is that Pd, with a much larger atomic size (hence larger compressibility) than the rest of the constituents, not only heightens the activation energy barrier ( E _a ) for interstitial motion by narrowing the diffusion channels but simultaneously also reduces E _a for vacancies due to less energy penalty required for bond length change between the initial and the saddle states. Our findings have a broad implication that the dynamics of point defects can be manipulated by taking advantage of the atomic size disparity, to facilitate point-defect annihilation that suppresses void formation and swelling, thereby improving radiation tolerance.