We study curvature perturbations in the anisotropic inflationary model with a complex scalar field charged under a $U(1)$ gauge field in Bianchi I universe. Due to Abelian Higgs mechanism, the gauge field receives an additional longitudinal mode. We verify that the dominant contributions into statistical anisotropies come from matter fields perturbations and one can neglect the contributions from the metric perturbations. It is shown that the contribution of longitudinal mode into the statistical anisotropy power spectrum, though exponentially small, has an opposite sign compared to the corresponding contribution from the transverse mode. We obtain an upper bound on gauge coupling in order to satisfy the observational constraints on curvature perturbations anisotropy. ; Comment: V3: The effects of gauge coupling on power spectrum are clarified, new results added, discussions improved, JCAP published version