In view of the problems of large tracking error of angular displacement and relatively serious vibration amplitude of the gear transmission system,the angular displacement and output torque produced in the process of straight tooth cylindrical gear movement were studied.The model of the gear transmission system was established,and the dynamic equation of gear transmission system was given.The traditional linear PID control system was improved and the nonlinear PID control system was designed.The weight coefficient of PSO was improved by using PSO.The improved particle swarm optimization algorithm was used to optimize the nonlinear PID control system,and the PID controller parameters were adjusted by the feedback error,so as to achieve the compensation goal of the control system output error.MATLAB software was used to simulate the output error and torque of the optimized gear transmission system,and the results were compared with that of linear PID control.The results show that compared with the linear PID control system,the optimized gear transmission control system can adjust all control parameters online,thus improving the accuracy of gear transmission.The control system has fast response speed,good stability,strong antiinterference ability and high control accuracy.