This study aimed to identify dietary patterns (DPs) and their associations with sociodemographic factors and diet quality in Portuguese adults and the elderly. Cross-sectional data were obtained from the National Food, Nutrition and Physical Activity Survey (2015–2016), with two non-consecutive dietary 24 h recalls. Food items were classified according to the NOVA system and its proportion (in grams) in the total daily diet was considered to identify DPs by latent class analysis, using age and sex as concomitant variables. Multinomial logistic and linear regressions were performed to test associations of DPs with sociodemographic characteristics and diet quality, respectively. Three DPs were identified: “Traditional” (higher vegetables, fish, olive oil, breads, beer and wine intake), “Unhealthy” (higher pasta, sugar-sweetened beverages, confectionery and sausages intake) and “Diet concerns” (lower intake of cereals, red meat, sugar-sweetened and alcoholic beverages). “Unhealthy” was associated with being younger and lower intake of dietary fiber and vitamins and the highest free sugars and ultra-processed foods (UPF). “Diet concerns” was associated with being female and a more favorable nutrient profile, but both DPs presented a higher contribution of UPF than the “Traditional” DP. These findings should be considered for the design of food-based interventions and public policies for these age groups in Portugal.