In order to tackle the problem of representing the distribution of complicated data using LDA, this paper proposes a novel method for constructing the non-parametric scatter matrix. Compared to classical LDA, our method can describe the classification boundary in a better way while preserving more useful information for classification. Since the non-parametric within-class scatter matrix may be singular for small sample-size problem, we propose a two-stage discriminant analysis method to optimize the criterion function. The human face images are projected onto the principal component subspace of the mixture scatter matrix via SVD so that the within-class scatter matrix in the projection subspace is singular. Via CS decomposition, we theoretically analyze the problem of solving the diagonal scatter matrix and prove that the projection matrix satisfies the orthogonally constraint. The experimental results on three face databases, i.e., the ORL database, the Yale database and the YaleB database, demonstrate the improvement of the proposed method over the traditional subspace methods.