Recent studies have revealed a strong correlation between the star formation rate (SFR) and stellar mass of the majority of star-forming galaxies, the so-called star-forming main sequence. An empirical modeling approach (the 2-SFM framework) that distinguishes between the main sequence and rarer starburst galaxies is capable of reproducing most statistical properties of infrared galaxies, such as number counts, luminosity functions, and redshift distributions. In this paper, we extend this approach by establishing a connection between stellar mass and halo mass with the technique of abundance matching. Based on a few simple assumptions and a physically motivated formalism, our model successfully predicts the (cross-)power spectra of the cosmic infrared background (CIB), the cross-correlation between CIB and cosmic microwave background (CMB) lensing, and the correlation functions of bright, resolved infrared galaxies measured by Herschel, Planck, ACT, and SPT. We use this model to infer the redshift distribution of CIB-anisotropies and of the CIB × CMB lensing signal, as well as the level of correlation between CIB-anisotropies at different wavelengths. We study the link between dark matter halos and star-forming galaxies in the framework of our model. We predict that more than 90% of cosmic star formation activity occurs in halos with masses between 1011.5 and 1013.5 M⊙. If taking subsequent mass growth of halos into account, this implies that the majority of stars were initially (at z > 3) formed in the progenitors of clusters (Mh(z = 0) > 1013.5 M⊙), then in groups (1012.5