The effects of ultrasonication pre-treatment of graphene oxide (GO) on the mechanical properties of GO/poly(vinyl alcohol) (PVA) composites were studied. Results show that the mechanical properties are sensitive to ultrasonication time (or energy input) and there is an optimum ultrasonication time (OUT) that leads to the largest improvement in mechanical property. If the ultrasonication time is less than OUT, the GO sheets are not fully exfoliated, and only a partial reinforcement effect is achieved. Ultrasonication times longer than OUT cause a reduction of GO sheet size and impair the efficiency of mechanical improvement. The optimized ultrasonication energy input of 15 W h/L may serve as a general guideline for preparation of GO composites.