We study the linear and nonlinear response of a unidirectional reflector where a nonlinear breaking of the Lorentz reciprocity is observed. The device under test consists of a racetrack microresonator, with an embedded S-shaped waveguide, coupled to an external bus waveguide (BW). This geometry of the microresonator is known as “taiji” microresonator (TJMR). Here, we show that a full description of the device needs to consider also the role of the BW, which introduces (i) Fabry-Perot oscillations (FPOs) due to reflections at its facets, and (ii) asymmetric losses, which depend on the actual position of the TJMR. At sufficiently low powers the asymmetric loss does not affect the unidirectional behavior, but the FP interference fringes can cancel the effect of the S-shaped waveguide. However, at high input power, both the asymmetric loss and the FPOs contribute to the redistribution of energy between counterpropagating modes within the TJMR. This strongly modifies the nonlinear response, giving rise to counter-intuitive features where, due to the FP effect and the asymmetric losses, the BW properties can determine the violation of the Lorentz reciprocity and, in particular, the difference between the transmittance in the two directions of excitation.