We study the transport of a single photon in two coupled one-dimensional semi-infinite coupled-resonator waveguides (CRWs), in which both end sides are coupled to a dissipative cavity. We demonstrate that a single photon can transfer from one semi-infinite CRW to the other nonreciprocally. Based on such nonreciprocity, we further construct a three-port single-photon circulator by a T-shaped waveguide, in which three semi-infinite CRWs are pairwise mutually coupled to each other. The single-photon nonreciprocal transport is induced by the breaking of the time-reversal symmetry and the optimal conditions for these phenomena are obtained analytically. The CRWs with broken time-reversal symmetry will open up a kind of quantum devices with versatile applications in quantum networks. ; Comment: 10 pages, 6 figures