Summary We propose several parsimonious models for higher order Markov chains, applied to the study of municipal rating migrations in credit risk. In full parameterized Markov chain models, the number of parameters increases very rapidly as the order in the Markov chain grows and this can yield biased estimates when certain sequences of states are rare. For some processes, as in the case of credit ratings, this problem is accentuated because the transitions between distant states are unlikely (persistent transitions). We introduce the short and long persistence models and compare them with the full parameterized Markov chain, achieving a better fit with a lower number of parameters. Furthermore, downgrade momentum effects are found in the rating process, which are consistent with recent empirical findings.