Exact 3D tracking of facial feature points is appealing for many applications in human-machine interaction. In this work a 3D Active Shape Model (ASM) that can be shifted, scaled, and rotated is used to track the points. The efficient Gauss-Newton method is applied to estimate the 3D ASM, rotation, translation, and scale parameters. If the head turns to one side, some points might be occluded but they are still considered for the estimation of the parameters. A robust error norm that reduces (or ideally cancels) the influence of occluded points is applied. With some algebraic transformations the computational cost per frame can be further reduced. The proposed algorithm is evaluated on the basis of the Airplane Behavior Corpus.