A phase field model is coupled to strain gradient crystal plasticity based on dislocation densities. The resulting model includes anisotropic plasticity and the size-dependence of plastic activity, required when plasticity is confined in region below few microns in size. These two features are important for handling microstructure evolutions during diffusive phase transformations that involve plastic deformation occurring in confined areas such as Ni-based superalloys undergoing rafting. The model also uses a storage-recovery law for the evolution of the dislocation density of each glide system and a hardening matrix to account for the short-range interactions between dislocations. First, it is shown that the unstable modes during the morphological destabilization of a growing misfitting circular precipitate are selected by the anisotropy of plasticity. Then, the rafting of γ′ precipitates in a Ni-based superalloy is investigated during [100] creep loadings. Our model includes most of the important physical phenomena accounted for during the microstructure evolution, such as the presence of different crystallographic γ′ variants, their misfit with the γ matrix, the elastic inhomogeneity and anisotropy, the hardening, anisotropy and viscosity of plasticity. In agreement with experiments, the model predicts that rafting proceeds perpendicularly to the tensile loading axis and it is shown that plasticity slows down significantly the evolution of the rafts.