Human mobility influenced the spread of the COVID-19 virus, as revealed by the high spatiotemporal granularity location service data gathered from smart devices. We conducted time series clustering analysis to delineate the relationships between human mobility patterns (HMPs) and their social determinants in California (CA) using aggregated smart device tracking data from SafeGraph. We first identified four types of temporal patterns for five human mobility indicator changes by applying dynamic-time-warping self-organizing map clustering methods. We then performed an analysis of variance and linear discriminant analysis on the HMPs with 17 social, economic, and demographic variables. Asians, children under five, adults over 65, and individuals living below the poverty line were found to be among the top contributors to the HMPs, including the HMP with a significant increase in the median home dwelling time and the HMP with emerging weekly patterns in full-time and part-time work devices. Our findings show that the CA shelter-in-place policy had varying impacts on HMPs, with socially disadvantaged places showing less compliance. The HMPs may help practitioners to anticipate the efficacy of non-pharmaceutical interventions on cases and deaths in pandemics.