Abstract: To solve the problem of insufficient consideration of sensor type factors and recognition methods in prediction methods of human activity recognition categories and accuracy, a human activity recognition method based on multi-modal sensors of smart phones was proposed, which includes inertial sensors, magnetometers and barometers. In addition, Stacking was used to fuse traditional random forest, support vector machine (SVM), K-nearest neighbor (KNN) and naive Bayesian algorithm, which forms an optimized human activity recognition classifier by learning training set data. Experiments show that the accuracy of the system is 99.0%, and the sensitivity and specificity of the system are 99.0% and 99.8% respectively. It can distinguish three similar movements including walking, upstairs and downstairs. Compared with the traditional single sensor activity recognition system, the system has the highest accuracy, sensitivity and specificity, 14.0%, 11.4% and 2.1% higher than the SVM algorithm; 3.4%, 3.3% and 2.0% higher than the KNN algorithm; and 1.8%, 2.0% and 0.6% higher than the random forest algorithm, respectively.