Because of the enormous range of time and space scales involved in dislocation dynamics, plastic modeling at macroscale requires a continuous formulation. In this paper, we present a rigorous formulation of the transition between the discrete, where plastic flow is resolved at the scale of individual dislocations, and the continuum, where dislocations are represented by densities. First, we focus on the underlying coarse-graining procedure. Our work reveals that both a spatiotemporal convolution and an ensemble average are required and that the emerging correlation-induced stresses are scale dependent. Each of these stresses can be expanded into the sum of two components. The first one depends on the local values of the dislocation densities and always opposes the sum of the applied stress and long-range mean field stress generated by the geometrically necessary dislocation (GND) density; this stress acts as a friction stress. The second component depends on the local gradients of the dislocation densities and is inherently associated to a translation of the elastic domain; therefore, it acts as a back stress. Finally, we show that that these friction and back stresses contain symmetry-breaking components that were missing in previous continuous formulations and that make, at mesoscale, the local stress experienced by dislocations depend on the sign of their Burgers vectors.