The irreversibility of a stationary time series can be quantified using the Kullback-Leibler divergence (KLD) between the probability of observing the series and the probability of observing the time-reversed series. Moreover, this KLD is a tool to estimate entropy production from stationary trajectories since it gives a lower bound to the entropy production of the physical process generating the series. In this paper we introduce analytical and numerical techniques to estimate the KLD between time series generated by several stochastic dynamics with a finite number of states. We examine the accuracy of our estimators for a specific example, a discrete flashing ratchet, and investigate how close the KLD is to the entropy production depending on the number of degrees of freedom of the system that are sampled in the trajectories.