In objective physical activity (PA) measurements, applying wider frequency filters than the most commonly used ActiGraph (AG) filter may be beneficial when processing accelerometry data. However, the vulnerability of wider filters to noise has not been investigated previously. This study explored the effect of wider frequency filters on measurements of PA, sedentary behavior (SED), and capturing of noise. Apart from the standard AG band-pass filter (0.29–1.63 Hz), modified filters with low-pass component cutoffs at 4 Hz, 10 Hz, or removed were analyzed. Calibrations against energy expenditure were performed with lab data from children and adults to generate filter-specific intensity cut-points. Free-living accelerometer data from children and adults were processed using the different filters and intensity cut-points. There was a contribution of acceleration related to PA at frequencies up to 10 Hz. The contribution was more pronounced at moderate and vigorous PA levels, although additional acceleration also occurred at SED. The classification discrepancy between AG and the wider filters was small at SED (1–2%) but very large at the highest intensities (>90%). The present study suggests an optimal low-pass frequency filter with a cutoff at 10 Hz to include all acceleration relevant to PA with minimal effect of noise.