Because of its significant impact on the precision and efficiency of mechanical machining, cutting chatter is one of the hot topics in the manufacturing regions. In order to prevent chatter, stability Lobe chart is usually used to choose appropriate machining parameters, and tool point frequency response functions (FRFs) of machine tools are essential in calculating the stability Lobe. Based on mode superposition method, excitation experiment and Receptance Coupling Substructure Analysis (RCSA), a new calculation method of tool point FRFs of machine tools was proposed. Firstly, superposition method and excitation experiment were used to get the end FRFs of the holder, the tool and the spindle part. Finally, RCSA was employed to obtain the tool point FRF by coupling the three substructures. Compared with the measured results, the proposed calculation method can obtain precise tool point FRF of vertical milling machine.