In this work, a coarse-grained (CG) model of carbon nanotube (CNT) reinforced polymer matrix composites is developed. A distinguishing feature of the CG model is the ability to capture interactions between polymer chains and nanotubes. The CG potentials for nanotubes and polymer chains are calibrated using the strain energy conservation between CG models and full atomistic systems. The applicability and efficiency of the CG model in predicting the elastic properties of CNT/polymer composites are evaluated through verification processes with molecular simulations. The simulation results reveal that the CG model is able to estimate the mechanical properties of the nanocomposites with high accuracy and low computational cost. The effect of the volume fraction of CNT reinforcements on the Young’s modulus of the nanocomposites is investigated. The application of the method in the modeling of large unit cells with randomly distributed CNT reinforcements is examined. The established CG model will enable the simulation of reinforced polymer matrix composites across a wide range of length scales from nano to mesoscale.