Low thermoplastic formability is a key factor limiting the usage of magnesium alloys, which otherwise can have broad application in automotive industry for their competitive strength to density ratio. Combining with experimental calibration and validation, we report a systematic numerical investigation about the plastic deformation of magnesium alloy AZ31B at different temperatures and subjected to different boundary conditions. By employing 3D Voronoi grains based microstructure and the crystal plasticity constitutive model developed by Staroselsky and Anand (2003), which accounts for both dislocation slip and deformation twinning in polycrystalline magnesium, we estimate the dependence of critical resolved shear stresses (CRSS) of different slip/twinning systems on temperature. We further obtain the fractional plastic strains contributed by individual slip/twinning systems at different loading conditions. Grain level deformation analysis indicates that there exists significant stress and plasticity inhomogeneity among grains.