Abstract: A set of coupled nonlinear differential equations of the FEM amplifier are derived in three dimensions for a configuration consisting of a cylindrical waveguide and a planar wiggler These equations include the effects of electron momentum spread and wiggler taper governing the self-consistent evolution of TE mode and trajectories of an ensemble of electrons. The initial conditions are chosen to model the adiabatic injection of a cylindrically symmetric electron beam. Computer program based on these equations is made and nonlinear effect including saturation efficiency and bandwidth is analyzed.