Abstract: In this paper, an anti mode mixing empirical mode decomposition (EMD) composite algorithm is proposed. First, the improved frequency heterodyne method is addressed to increase the spectral distance between mode mixing components in order to isolate the octave mode mixing components. Second, the wavelet singularity detection technology is applied to determine the adaptive sliding analysis window for each mode and then implement the intermittent mixing modes separation and their nonstationary parameters identification. In order to verify the validity of the algorithm, a test case and a grid simulation case are used to test and analyze the EMD composite algorithm. The analysis result shows that the proposed algorithm can isolate the mode mixing component effectively. Moreover, for a low frequency oscillation signal with complex modes, the proposed algorithm is able to isolate the corresponding low frequency oscillation mode and has high detection accuracy of oscillation mode information.