Abstract: In order to increase the convergent speed and to improve the overall searching ability of the algorithm,a Particle Swarm Optimization (PSO) method is proposed with adaptive inertia weight by the change of the number of iterations based on the analysis of inertia weight global best fitness of the PSO. The improved PSO increases the ability to avoid local optimum. Then a speaker recognition method using this improved algorithm to train Support Vector Machine (SVM) is presented. The experimental results show that the presented SVM method optimized by PSO for speaker recognition can achieve higher recognition accuracy and higher recognition speed.