The phosphorylation-dephosphorylation cycle is a common motif in cellular signaling networks. Previous work has revealed that, when driven by a noisy input signal, these cycles may exhibit bistable behavior. Here, a recently introduced theorem on network bistability is applied to prove that the existence of bistability is dependent on the stochastic nature of the system. Furthermore, the thermodynamics of simple cycles and cascades is investigated in the stochastic setting. Because these cycles are driven by the ATP hydrolysis potential, they may operate far from equilibrium. It is shown that sufficient high ATP hydrolysis potential is necessary for the existence of a bistable steady state. For the single-cycle system, the ensemble average behavior follows the ultrasensitive response expected from analysis of the corresponding deterministic system, but with significant fluctuations. For the two-cycle cascade, the average behavior begins to deviate from the expected response of the deterministic system. Examination of a two-cycle cascade reveals that the bistable steady state may be either propagated or abolished along a cascade, depending on the parameters chosen. Likewise, the variance in the response can be maximized or minimized by tuning the number of enzymes in the second cycle.