Abstract: Based on the theories of generalized transmission line and electron cyclotron resonance maser, the TE_{32, 12} mode is selected as the operating mode of a 170 GHz megawatt gyrotron by investigating the mode spectrum and studying the beam-wave coupling coefficient and the start-current of the modes carefully. Through appropriately arranging the parameters of the beam, the transient modes in the high order mode resonator are suppressed effectively. And through designing the gradual sections of a kind of taped cavity, the parasitic modes are suppressed furthest. As a result, an output power of 1.8 MW, corresponding to 50% efficiency, and an oscillation frequency of 170.18 GHz have been achieved with a pitch factor 1.3, radius of guiding center 9.4mm, 80 kV, 45A helical electron beam at a guiding magnetic field of 6.715T. At the same time, we have also set up a velocity-spread model suitable to actual circumstances and discussed the performance impact to the gyrotron.