We consider models of inflation with U(1) gauge fields and charged scalar fields including symmetry breaking potential, chaotic inflation and hybrid inflation. We show that there exist attractor solutions where the anisotropies produced during inflation becomes comparable to the slow-roll parameters. In the models where the inflaton field is a charged scalar field the gauge field becomes highly oscillatory at the end of inflation ending inflation quickly. Furthermore, in charged hybrid inflation the onset of waterfall phase transition at the end of inflation is affected significantly by the evolution of the background gauge field. Rapid oscillations of the gauge field and its coupling to inflaton can have interesting effects on preheating and non-Gaussianities. ; Comment: minor changes, references added, figures are modified, conforms JCAP published version