With the continuous improvement of manufacturing capability, the automation degree of product production process is getting higher and higher. For traditional visual recognition algorithm, it needs a lot of technical and professional knowledge to model visual recognition. Moreover, the designed algorithm can only adapt to a single working condition and has no generality, while deep learning algorithm is a general learning algorithm, which can recognize different kinds of targets through training. Based on this, an end-to-end training recognition algorithm was designed for identifying the number of axle end in the process of traction shaft processing in an enterprise. The system obtained the target image by high precision industrial camera, and input the original RGB image into the network without preprocessing. The target was recognized by deep learning algorithm. The experimental results show that the designed algorithm has good robustness and high accuracy.