Transient elasticity (TE) is a concept useful for a systematic generalization of viscoelasticity. Due to its thermodynamic consistency, it naturally leads to a simple description of non-Newtonian effects displayed by polymeric fluids, granular media, and other soft matter. We employ a continuum-mechanical theory that is derived from TE and tailored to polymeric fluids, showing how it captures a surprisingly large number of phenomena in shear and elongational flows, including stationary, oscillatory, and transient ones, as well as the flow down an inclined channel. Even the Weissenberg effect is well accounted for. This theory is applicable for small- as well as large-amplitude deformations. We concentrate on the former in the present article, leaving the latter to a companion article.