The controllability of bilinear systems is well understood for finite dimensional isolated systems where the control can be implemented exactly. However when perturbations are present some interesting theoretical questions are raised. We consider in this paper a control system whose control cannot be implemented exactly but is shifted by a time independent constant in a discrete list of possibilities. We prove under general hypothesis that the collection of possible systems (one for each possible perturbation) is simultaneously controllable with a common control. The result is extended to the situations where the perturbations are constant over a common, long enough, time frame. We apply the result to the controllability of quantum systems. Furthermore, some examples and a convergence result are presented for the situation where an infinite number of perturbations occur. In addition, the techniques invoked in the proof allow to obtain generic necessary and sufficient conditions for ensemble controllability.