All children, including those living with disabilities, have the right to be provided with Inclusive Education (IE) together with appropriate support in scholarly settings. A key factor for educational inclusion is peers’ attitudes towards disabilities, which impact disabled students’ social participation and learning. Physical Education (PE) classes represent an opportunity for students with disabilities to get psychological, social, health, and educational benefits. This study aimed to assess Spanish students’ attitudes regarding their peers with disabilities during PE lessons and to explore possible differences between gender, school location, and age group. The sample consisted of 1437 students from public schools in Extremadura (Spain) from the primary and secondary stages. Participants completed the Attitude towards Students with Disabilities in Physical Education (EAADEF-EP) Questionnaire. The Mann-Whitney U test was carried out to establish differences in scores according to sex, school location, age group, and correlations between age and item scores using the Spearman’s Rho test. The results displayed significant differences in the total and item scores considering sex and centre location, with good reliability values (Cronbach’s alpha = 0.86). The EAADEF-EP Questionnaire has proven to be a quick, easy, and inexpensive tool to assess attitudes. Girls and participants whose school was in a rural setting had better attitudes towards inclusion. This study’s results highlight the importance of carrying out educational actions and programs to improve students’ attitudes towards their peers with disabilities considering the influences of the studied variables.