Focused on needs of the intelligent monitoring of moving materials, a visual tracking method of multiple moving objects is proposed. In detection of moving objects, edge detection based three frames difference was combined with background subtraction, the integrality of the detected objects outline was improved, meanwhile the cavitation of the objects and the image noise were reduced, so the more effective data foundation was provided for moving objects tracking. The detection, segmentation, classification and marker for target areas were based on mathematical morphology and the area threshold denosing, and Kalman filter prediction was further combined for tracking and positioning of multiple moving objects simultaneously. The experiment results show that the proposed method can visually detect and track multiple materials precisely, meanwhile the reliability and realtime performance requirements are guaranteed.