Graphene oxide (GO) was chemically functionalized to prepare polyurethane (PU) composites with improved mechanical and thermal properties. In order to achieve a well exfoliated and stable GO suspension in an organic solvent (dimethylformamide, DMF), 4, 4′-methylenebis(phenyl isocyanate) and polycaprolactone diol, which were the two monomers for synthesizing PU, were selectively used to functionalize GO. The obtained functionalized GO (FGO) could form homogeneous dispersions in DMF solvent and the PU matrix, as well as provide a good compatibility with the PU matrix. The most efficient improvement of mechanical properties was achieved when 0.4 wt.% FGO was added into the PU matrix, showing increases in the tensile stress, elongation at break and toughness by 34.2%, 27.6%, and 64.5%, respectively, compared with those of PU. Regarding the thermal stability, PU filled with 1 wt.% FGO showed the largest extent of improvement with T2% and T50% (the temperatures at which 2% and 50% weight-loss happened) 16 °C and 21 °C higher than those of PU, respectively. The significant improvement in both mechanical properties and thermal stability of FGO/PU composites should be attributed to the homogeneous dispersion of FGO in the PU matrix and strong interfacial interaction between them.