Aimed at the problem that the operating conditions of the wind turbine gearbox were complex and the background noise was large, and the fault characteristic information was difficult to extract, a fault feature extraction method for wind turbine gearbox based on variational mode decomposition (VMD) and fractional Hilbert transform (FHT) was proposed. The fault signals of the gearbox were decomposed by using VMD,and a decomposition quality factor was defined to select the optimal decomposition layer number K of VMD. Fractional Hilbert transform was performed for the each modal component decomposed by the optimized VMD, and the marginal spectrum of the each modal component was calculated and linearly superposed. The marginal spectral frequency domain feature was extracted as the feature value of the gearbox fault signal. The experimental results show that by using the method, the fault characteristics of the fan gearbox can be extracted accurately and better fault identification effect can be obtained