Abstract: A series of studies on speaker recognition algorithm based on relevance vector machine (RVM) and gaussian mixture model (GMM) was proposed in this paper. The sparseness and probability prediction of RVM make the algorithm suitable for speaker recognition in applications. The robust speech features based on GMM are investigated. In contrast to the most current systems based on frame-level discrimination, the approach has two outstanding merits. The first is the system provides direct discrimination between whole sequences by combining GMM as underlying generative models in feature-space. The paper focused on two main feature space: mel-frequency cepstrum coefficient (MFCC) and instantaneous frequencies (IF). The second combines the high generalization, kernel tricks, and sparser performance of RVM to generate more robust classification results and to reduce the computational complexity. The simulations using the Chains database and the AHUMADA database show that the proposed algorithm outperforms the other systems on reducing the relative error rates and reducing the computational complexity in high dimensionality space and big scale data.