A description of a new species of plant parasitic nematodes of the genus Rotylenchus from the family Hoplolaimidae is given and a recognition of Rotylenchus jaeni comb. n., previously known as subspecies R. magnus jaeni, as separate species is proposed. Rotylenchus montanus sp. n. is characterized by a hemispherical lip region with six rarely seven annuli, stylet 33–37 µm, female tail rounded, regularly annulated tip with 12–18 annuli and phasmid located 2–9 annuli anterior to anus. Rotylenchus montanus sp. n. is close to species of the monosexual group R. arsenjevi, R. corsicus, R. fragaricus, R. helicus, R. indorobustus and R. neorobustus, by a number of specific characteristics resulting from its specific matrix code: A5, B1, C1, D4, E2, F2, G3, H2, I2, J2, K2. Molecular characterization of R. montanus sp. n. and other Rotylenchus species are provided using D2–D3 expansion segments of 28S and the ITS1 of rRNA genes. The D2–D3 of 28S rRNA and the ITS1–rRNA sequences of R. montanus sp. n. differed in one nucleotide and in 16–20 nucleotides from those of an unidentified Rotylenchus species from Russia, respectively. Molecular analysis of populations of R. magnus and R. jaeni comb. n. demonstrated differences in the D2–D3 and the ITS1–rRNA sequences. These genetic differences together with some minor morphological characters support that both subspecies should be considered as two cryptic sibling species and warranted their elevation to species rank. The result of phylogenetic analysis of Hoplolaimidae for 45 sequences of the D2 and D3 expansion regions of 28S rRNA gene using Bayesian inference analysis under the complex model is presented. Phylogenetic tree of Rotylenchus species represents seven moderate to highly supported lineages. Grouping of Rotylenchus species within other hoplolaimids and analysis of phylogenetic relationships within the genus Rotylenchus using the ITS1 of rRNA gene sequences are also discussed.