Electro hydrostatic actuators (EHAs) are widely used in aircrafts and robotic systems, and their application is gradually progressing toward large mechanical equipment. In this paper, a new structure wherein an accumulator is attached to the pistonside of the EHA is proposed, which is used to achieve rapid startup under noload. In addition, a new parameter adaptive control method (PACM) is proposed to allow the accumulator to absorb the pulsation of the system pressure and flow rate in the best state when the external load is suddenly applied or changed. The mathematical models of EHA and PACM are established. The feasibility and validity of the proposed structure and PACM are then demonstrated. The simulation results demonstrate that the startup time of the EHA with the conventional hydraulic structure is 62.8 s, whereas that of our proposed EHA equipped with an accumulator is 2.1 s, which shortens the EHA startup time by 96.6%.The accumulator with PACM can reduce the pressure pulsation by 31%, when the external load of 5 000 N is applied. Moreover, the flow rate curve is excessively smoother and the PACM parameter can be conveniently adjusted according to specific working conditions.