We present an experimental study of the current-driven polarization modulation properties of VCSELs. In some VCSELs, abrupt polarization switching (PS) between two polarization modes is observed at a particular value of the pump current. We investigate the dynamics and the associated dominating time scales of PS as these features are strongly linked with the underlying physical mechanism causing the PS. To this end, we measure both for gain- and index-guided VCSELs the critical modulation amplitude necessary to steadily force PS back and forth across the PS point as a function of the modulation frequency. This yields the current-driven polarization modulation frequency response, which we compare with the thermal frequency response of the studied devices. The dynamic behavior turns out to be strikingly different for the different VCSEL types. Thermal effects only play a minor role in the PS in our index-guided VCSELs, while they really seem to lie at the origin of PS in the gain-guided VCSELs. By implementing this in a rate-equation based theoretical model, we are able to explain the peculiarities of the measured response curves and to reproduce the experimental findings.