We revisit the idea of generating primordial anisotropies at the end of inflation in models of inflation with gauge fields. To be specific we consider the charged hybrid inflation model where the waterfall field is charged under a U(1) gauge field so the surface of end of inflation is controlled both by inflaton and the gauge fields. Using delta N formalism properly we find that the anisotropies generated at the end of inflation from the gauge field fluctuations are exponentially suppressed on cosmological scales. This is because the gauge field evolves exponentially during inflation while in order to generate appreciable anisotropies at the end of inflation the spectator gauge field has to be frozen and scale invariant. We argue that this is a generic feature, that is, one can not generate observable anisotropies at the end of inflation within an FRW background. ; Comment: V3: new references added, JCAP published version