In a number of stochastic systems the random forcing is represented as a dichotomous Markov noise. A common characteristic of these models is that the noise is usually supposed to be independent of the state of the forced dynamical system. However, there are several situations in which positive or negative feedback exist between the system and the random driver. This paper investigates a class of systems characterized by feedback between dichotomous Markov noise and the system's dynamics. The effect of the feedback is accounted for through a state dependency in the transition rates of the dichotomous noise. We study noise-induced transitions in these systems, with special attention to the delicate problem of correctly defining the deterministic counterpart of the stochastic system. We find that (i) if in the absence of any feedback the dynamical system has a single deterministic stable point, the deterministic dynamics remain monostable when a negative feedback is introduced, while they may become bistable in the presence of a positive feedback. (ii) Noise may induce bistability in the presence of a null or negative feedback. (iii) Bistable deterministic dynamics, induced by the positive feedback, may be destroyed by the noise, which tends to stabilize the system around a new intermediate stable state between those of the deterministic dynamics.