Based on the analysis of the transformation relationship of coordinate systems in the visual guidance of the robot, a nonlinear calculation method for calibrated parameters based on the maximum likelihood estimation was proposed by constructing the matrix transformation measure function. Levenberg-Marquardt algorithm, widely used to solve the nonlinear least squares problem, was applied to solve parameters for camera calibration and handeye calibration at one time. Through the numerical simulation and the insite calibration, the results were compared with those from classical method. The results show that the proposed algorithm has reduced the estimation error and the noise sensitivity and improved stability compared with the classical algorithm.