Mobility scooters are three or four-wheeled electric-motor-driven wheelchairs with a handlebar for steering. In the use environment of a mobility scooter, there may be many obstacles that need to be avoided, such as pedestrians on a narrow sidewalk or cars on the edge of the road. These obstacles not only interfere with smooth driving but also induce mental strain on the user such as fear and anxiety, which may cause a decrease in comfort and an accident. The purpose of this study is to estimate physical obstacles in the road environment when driving a mobility scooter from operation information and physiological information of the user obtained during driving. We measured skin conductance, manipulation amount of accelerator, steering torque, and acceleration when driving on a course with physical obstacles. We tried to build a model that estimates the type of physical obstacles in the course based on these time-series data by using a random forest, which is one of the machine learning methods.