Glass fiber reinforced polymer composites (GFRP) are widely used in light structures due to their special material properties. However, excessive tool wear will occur during the processing, which directly affects the integrity of the workpiece materials. Based on this, an online monitoring criterion of tool wear failure in GFRP material processing is presented. The vibration signal and spindle power signal of tool failure were collected, the cutting experiments of tool and workpiece were carried out, the measured vibration signal was analyzed, and the monitoring criterion was verified by experiments. Furthermore, the tool chatter monitoring criterion was established. The tool failure is analyzed from the chatter signal, and the feasibility of the criterion is verified by experiments.