Abstract:In view of the lacking of 3D posture information in 2D image recognition, while a large number of point clouds need tobe dealt in traditional 3D vision and computing time is long,a robot destacking, sorting and palletizing strategy was proposed based on the combination of improved Mask R-CNN and local point cloud iterative optimization. The Mask R-CNN network was improved, and the spatial transformer networks(STN)module was added after the ROIAlign structure to improve the recognition accuracy.The improved Mask R-CNN network was used to segment instance, and it was combined with the scene point cloud to obtain the object region of interest (ROI) scene local point cloud. The interactive closest point (ICP) algorithm with K-dimensional tree neighborhood search was used to register the local point cloud of the object ROI scene with the template point cloud. Finally, the pose estimation result was obtained. UR5 cooperative robot solved the automatic destacking, sorting and palletizing problems according to the results.The experimental results show that using the improved Mask R-CNN network improves the accuracy of target recognition, using the ROI local point cloud method reduces the iteration times of scene point cloud and template point cloud registration, and improve the efficiency of industrial robot’ s destacking, sorting and palletizing.