In this work, the mechanical behavior of sparsely cross-linked, neutral polyacrylamide (PAAm) hydrogels containing densely cross-linked polyelectrolyte microgels of poly(2-acrylamido-2-methylpropanesulfonic sodium) (PNaAMPS) were studied systematically by varying the formulations. The microgel-reinforced (MR) hydrogels have a two-phase composite structure, where the disperse phase is the rigid double-network (DN) microgels, and the continuous phase is the soft PAAm matrix. At the optimal formulation, the MR gels showed high mechanical strength and toughness, comparable to conventional DN hydrogels. The two critical parameters for the substantial enhancement of mechanical strength and toughness of MR gels are the concentration of PNaAMPS microgel and the molar ratio of the PAAm to the PNaAMPS in the microgel phase. Selective dyeing of the embedded microgels in MR gels allowed for visualization of the deformation of microgels, and we found that the local strain of microgels was much smaller than the global strain applied on MR gels; this indicates that isostress model (Reuss’s model) is more suitable than isostrain model (Voigt’s model) for this composite system.