Abstract: Aiming at the deficiencies of PID control method and visual servo control method commonly used in the tracking scene of Rotor UAV(unmanned aerial vehicle), this paper attempts to combine visual servo control with reinforcement learning, and proposes an intelligent tracking method for Rotor UAV based on reinforcement learning. Firstly, image-based visual servo is used to track the closed-loop control of the Rotor UAV, and then a reinforcement learning model is established to adjust the servo gain with Sarsa learning algorithm. After many training sessions, the Rotor UAV can choose its own visual servo gain. In this paper, the experiment of tracking the moving target of Rotor UAV in physical and simulation scenarios is designed. The experimental results demonstrate that the proposed method has better tracking effect than PID control and classical image-based visual servo control method.