This paper presents a control scheme of pneumatic servo systems for practical use, in which a two-layer neural network is used to construct the inverse system of the plant, and disturbance to the plant is estimated. The influence of the disturbance is eliminated by subtracting the estimated disturbance from the output of the controller. To improve the learning ability of the NN, σ modification method, which is one of the robust parameter adjusting methods of the robust adaptive control, is introduced. To confirm the effectiveness of the proposed control scheme, experiments using an existent pneumatic servo system were conducted. The experimental results showed that the external force was estimated well by the disturbance estimation mechanism, and the influence of the external force to the plant output was eliminated immediately after the external force was applied. In addition, high-speed learning of NN could be realized using the switching σ modification method.