Quantitative evaluation of motor function is of great demand for monitoring clinical outcome of applied interventions and further guiding the establishment of therapeutic protocol. This study proposes a novel framework for evaluating upper limb motor function based on data fusion from inertial measurement units (IMUs) and surface electromyography (EMG) sensors. With wearable sensors worn on the tested upper limbs, subjects were asked to perform eleven straightforward, specifically designed canonical upper-limb functional tasks. A series of machine learning algorithms were applied to the recorded motion data to produce evaluation indicators, which is able to reflect the level of upper-limb motor function abnormality. Sixteen healthy subjects and eighteen stroke subjects with substantial hemiparesis were recruited in the experiment. The combined IMU and EMG data yielded superior performance over the IMU data alone and the EMG data alone, in terms of decreased normal data variation rate (NDVR) and improved determination coefficient (DC) from a regression analysis between the derived indicator and routine clinical assessment score. Three common unsupervised learning algorithms achieved comparable performance with NDVR around 10% and strong DC around 0.85. By contrast, the use of a supervised algorithm was able to dramatically decrease the NDVR to 6.55%. With the proposed framework, all the produced indicators demonstrated high agreement with the routine clinical assessment scale, indicating their capability of assessing upper-limb motor functions. This study offers a feasible solution to motor function assessment in an objective and quantitative manner, especially suitable for home and community use.