A fusion recognition algorithm involving an expert system (ES) and a multimodal long short-term memory (MLSTM) neural network was designed to solve the fault diagnosis problem of a large-scale hydraulic press, improve the accuracy of fault recognition, and ensure normal and effective operation of the hydraulic system. Firstly, through the data acquisition system of the large hydraulic press, the hydraulic press pressure, solenoid valve and travel switch state signals were obtained, and the data were digitally filtered and cleaned to obtain a 22 dimensional feature vector. An LSTM model was constructed using the 22-dimensional eigenvector, and an optimal numbers of input nodes, hidden layer nodes, and output nodes were selected. The recognition rate under different training samples and the influence of feature vector dimension on the recognition rate were analyzed.The factors affecting the recognition rate of the LSTM model were analyzed, a method for using multiple parameter models for the same LSTM structure was proposed, and the ES was used to manage the parameter model to improve the recognition rate. The reasoning knowledge model, data cleaning knowledge model and scheduling knowledge model of multi-mode deep learning network (MLSTM) of expert system were designed. Further, a machine reasoning system that simplified feature vector data was designed, and the multi-modal LSTM (MLSTM) network learning training was completed using these data. During fault prediction and classification, the preliminary candidate results were obtained by reasoning through the ES, and the predicted results were sorted based on probability. The first N results of sorting were taken out, and the MLSTM was used for discrimination, thus the recognition time was effectively reduced. The mode conversion of MLSTM was achieved by using the reasoning function of the ES, and the classification accuracy was improved considerably. In the system, 12 fault classes, 120 training samples, and 1 920 test samples were used for testing. The recognition rate of the ES-MLSTM is 100%, while that of SVM is 92.9%, PSOSVM is 96.3%, and BP is 73%, which proves that the recognition method based on ES-MLSTM meets the requirements of fault diagnosis.