Mechanisms that electrons are directly accelerated by the laser-plasma interaction in non-resonant cases are studied. First, by use of a linearly polarized Gaussian laser beam, a three-dimensional model is presented to demonstrate that the frequency and the amplitude of electron oscillations can be significantly modulated by the transverse ponderomotive force, within the confinement of an underdense plasma channel. On the one hand, the transverse ponderomotive force can felicitously make electrons to experience the large amplitude oscillations and push them to the regions at a low dephasing rate. On the other hand, when the electrons oscillate across the channel with small amplitudes, the dephasing rate also can be effectively reduced by the nonlinear modulation arising from the transverse ponderomotive force. These kinds of modulations can lead electrons to stay in phase with the laser field for a longer time and thus enhance their energy gain, which also enables the mechanism of transverse ponderomotive modulation being in direct laser acceleration. This mechanism is determined by the plasma density and the laser intensity and radius. Detailed numerical results are also given which show that the electron acceleration induced by this ponderomotive modulation quite distinguishes from the parametric instability and the resonance from a driving force. Moreover, a theoretical model for the parametric amplification, which makes up the restriction of the quasi-two-dimensional model, is provided to verify that non-resonant direct laser acceleration can come from the parametric instability in the three-dimensional case.