Abstract:As realtime system Jacobian Matrix is needed in traditional visual servo control system of robot to calculate the nonlinear relationship between space of an image and of the robot, by this method the complexity of this system structure, large amount of calculation, and its sensitivity to system model error are expected to improve. Aimed at the problems, based on analysis of the mathematical model of the visual servo control system, a radial base function (RBF) neutral network controller was designed to realize visual servo control in imagebased robot with coordination function of eyes and hands, therefore control algorithm was simplified. Through following simulation experiment, it was proved that the designed control system of RBF neutral network having good target positioning effect and generalization capability. The time needed in the visual servo control is greatly shortened, and the control system in real time is very well ensured.