15 pages, 12 figures ; We review the theory of the Casimir effect using scattering techniques. After years of theoretical efforts, this formalism is now largely mastered so that the accuracy of theory-experiment comparisons is determined by the level of precision and pertinence of the description of experimental conditions. Due to an imperfect knowledge of the optical properties of real mirrors used in the experiment, the effect of imperfect reflection remains a source of uncertainty in theory-experiment comparisons. For the same reason, the temperature dependence of the Casimir force between dissipative mirrors remains a matter of debate. We also emphasize that real mirrors do not obey exactly the assumption of specular reflection, which is used in nearly all calculations of material and temperature corrections. This difficulty may be solved by using a more general scattering formalism accounting for non-specular reflection with wavevectors and field polarizations mixed. This general formalism has already been fruitfully used for evaluating the effect of roughness on the Casimir force as well as the lateral Casimir force appearing between corrugated surfaces. The commonly used `proximity force approximation' turns out to lead to inaccuracies in the description of these two effects.