Background: This cross-sectional study analyzed the association of leisure-time physical activity (PA) and sedentary behavior (SB), nonleisure PA and SB, and total PA and SB in different time segments of the day with mental health among Dutch adolescents and young adults. Methods: A total of 881 participants aged 16–25 years completed an online survey. Mental health was assessed using the Mental Health Inventory-5, and participants also reported sex, age, and income. They filled out a questionnaire of types of PA and SB for each hour of the day. Activities were categorized into nonleisure and leisure, during the morning, afternoon, evening, and for the whole day. Results: Participants (52.8% female, on average 20.8 y) generally engaged in more leisure-time PA and SB during weekends compared with weekdays, and more nonleisure activities on weekdays. Associations varied between time segments and days of the week. Positive associations of leisure-time and total PA during the whole day and evenings with mental health were observed on weekdays. Total, leisure-time, and nonleisure-time SB were associated with worse mental health. Nonleisure PA was not associated with mental health. Conclusions: Leisure-time PA was found to have a favorable association with mental health, particularly in the evenings of weekdays and afternoons of weekend days. On the other hand, leisure SB was associated with poorer mental health in most of the time segments analyzed, and nonleisure SB in the evenings was also related to worse mental health. The type and timing of PA and SB behaviors play an important role in the relationship with mental health.