Ultrafast third-order optical nonlinearity of Ge–Ga–Ag–S chalcogenide glasses at the wavelength of 820 nm has been measured using femtosecond time-resolved optical Kerr (OKE) technique. The results show that Ge–Ga–Ag–S glasses have large third-order optical nonlinear susceptibility, χ(3) and the response time is also subpicosecond, which are predominantly due to the ultrafast distortion of electron cloud surrounding the balanced positions of Ge, Ga, Ag and S atoms. What’s more, a strong dependence of χ(3) on the composition and microstructure of these glasses was found which shows that [GeS4] and [GaS4] tetrahedra play an important role on the third-order optical nonlinearity. These Ge–Ga–Ag–S chalcogenide glasses would be expected as promising materials applied on all-optical switching devices.