In this paper, we study the outage performance of a complex system consisting of a free-space optical (FSO)/radio-frequency (RF) link. Using radio over free-space optics technology, the FSO link carries Worldwide Interoperability for Microwave Access (WiMAX) signals from a core network to a WiMAX base station, delivering traffic to multiple end RF users. A novel closed-form analytical expression for overall outage probability is derived when M-ary phase-shift keying (M -PSK) and M-ary quadrature amplitude modulation (M-QAM) are applied. The analysis is performed when the FSO link is under the influence of the Gamma–Gamma turbulence, path loss, and misalignment between the transmitter and receiver apertures, and the RF part is influenced by the Gamma-shadowed Nakagami-m multipath fading. To illustrate the usefulness of the derived expressions, we present some numerical results that enable us to estimate the effects of different transceiver and channel parameters on the outage probability. The results are used for optimizing the transmitter laser beam radius at the waist to achieve the minimal overall outage probability in the case of different conditions over the RF part. The numerical and simulation results show that multipath fading severity and shadowing spread over the RF part have a significant effect on the optimal value of the laser beam waist and can decrease the overall outage probability for several orders of magnitude.