In this paper, we present new theoretical and experimental results on the elliptical instability in a liquid metal contained in a rotating deformable cylinder in the presence of an imposed magnetic field. The imposed field, which is aligned with the rotating axis, has a double interest. On the one hand, it permits an analysis of the inertial waves excited by the elliptical instability by measuring their induced magnetic fields. On the other hand, it permits the control of the instability by acting on the Joule damping. In this paper, firstly an analytical calculation of the magnetic field induced by the flow and its associated Joule damping is presented. Also, the linear and weakly nonlinear theories of the elliptical instability are extended to include magnetic field effects. Then, the description of the experiments starts by the presentation of the effect of the imposed magnetic field strength. Close to the instability threshold, both super- and subcritical bifurcations are identified. When the imposed field is decreased, we observe a transition towards complex nonlinear evolutions that we describe with the help of two-dimensional phase diagrams. In a second set of experiments, we vary the eccentricity of the elliptic deformation over a large range in order to demonstrate that far from the instability threshold, the mean inertial wave amplitude is uncorrelated to the eccentricity. We show that, for a given eccentricity, this mean amplitude decreases when the rotation rate increases. In a last series of experiments, we focus on the description of the nonlinear evolution of an oscillatory eigenmode which is different from the principal stationary mode.