In multivariate statistical process monitoring (MSPM), isolation of faulty variables is a critical step that provides information for analyzing causes of process abnormalities. Although statistical fault detection has received considerable attention in academic research, studies on multivariate fault isolation are relatively fewer, because of the difficulty in analyzing the influences of multiple variables on monitoring indices. The commonly used tools for fault isolation, such as contribution plots, reconstruction-based methods, etc., have several shortcomings limiting their implementation. To solve the problems of the existing methods, this paper reveals the relationship between the problems of multivariate fault isolation and variable selection for discriminant analysis. Furthermore, by revealing the equivalence between discriminant analysis and regression analysis, the problem of multivariate fault isolation is further formulated in a form of penalized regression which can be solved efficiently using state-of-the-art algorithms. Instead of offering a single suggested set of faulty variables, the proposed method provides more information on the relevance of process variables to the detected fault, facilitating the subsequent root-cause diagnosis step after isolation. The benchmark Tennessee Eastman (TE) process is used as a case study to illustrate the effectiveness of the proposed method.