A two rounds of self-balance robot system was designed by using STC11L04E and Freescale Kinetis 60, which was controlled by speech recognition. Speech recognition module was mainly composed of STC11L04E and LD3320, using a real-time speech recognition algorithm, the realization of speaker-independent speech control of self balance robot was achieved. To achieve the self balancing function of the robot, the triaxial acceleration sensor MMA7260 and gyroscope ENC-03M were used to acquire the acceleration and angular velocity values in real-time. Furthermore, in order to determine the robot’s state, Freescale Kinetis 60 (Master control chip) was used for data fusion. To keep the robot in a vertical state, the master control chip control motor positive and negative rotation by PID algorithm. Experimental results demonstrate the robot can realize self balance in short time, and can complete forward, backward, turn left, turn right, acceleration, deceleration according to speech commands.