The deformation behaviors of A356 alloy with various initial grain structures, from a very coarse dendrite to a fine equiaxed structure, prepared at different pouring temperatures, from 625°C to 725°C, were investigated. Cylindrical specimens with 12 mm in diameter and 10 mm in height were compressed in the semisolid state to a height reduction of 6 mm at an initial strain rate of 1.39 × 10−3/s and then jumped to 1.39 × 10−2/s. The instantaneous strain rate sensitivity was calculated under iso-structure conditions. Three very different deformation behaviors have been observed, which are believed to have resulted from their different starting microstructures. The materials produced at low pouring temperatures evolve into a fine globular structure after isothermal holding at semisolid region and shows a very low compression stress, while the materials cast at high pouring temperatures remain as coarse and dendritic in its microstructure and have much higher compression stress. The macrostructural breakage and microstructural segregation of hot compressed samples were analyzed, and the deformation mechanisms were discussed. The low temperature poured material is not sensitive to an instantaneous change in strain rate, while the high temperature poured material is very sensitive.