Different aspects of the daily movement pattern—sitting, light intensity physical activity, and moderate- and vigorous intensity physical activity—have each independently been associated with health and longevity. Previous knowledge of the amount and distribution of these aspects in the general Swedish population, as well as the fulfilment rate of physical activity recommendations, mainly relies on self-reported data. More detailed data assessed with objective methods is needed. The aim of the study was to present descriptive data on the daily movement pattern in a middle-aged Swedish population assessed by hip-worn accelerometers. The cohort consisted of 948 participants (51% women), aged 50 to 64 years, from the Swedish CArdioPulmonary bioImage pilot Study. In the total sample, 60.5% of accelerometer wear time was spent sitting, 35.2% in light physical activity and 3.9% in moderate- and vigorous physical activity. Men and participants with high educational level spent a larger proportion of time sitting, compared to women and participants with low educational level. Men and participants with a high educational level spent more time, and the oldest age-group spent less time, in moderate- and vigorous physical activity. Only 7.1% of the study population met the current national physical activity recommendations, with no gender, age or education level differences. Assessment of all three components of the daily movement pattern is of high clinical relevance and should be included in future research. As the fulfilment of national physical activity recommendations is very low and sitting time is very high in our middle-aged population, the great challenge remains to enhance the implementation of methods to increase the level of physical activity in this population.