Ultrafine-grained specimens with the average grain size of less than 2µm and medium-grained specimens were prepared from steel plates produced by an advanced thermo-mechanical control process. The smooth specimens were fatigued under axial tension compression at room temperature in air. The fatigue crack initiation process was investigated by orientation imaging microscopy based on EBSP in combination with atomic force microscopy and scanning electron microscopy. The results show that fatigue cracks initiated in the simple slip lines in the medium-grained specimens. On the other hand, in the ultrafine-grained specimens, complex slip deformation was formed in the vicinity of the grain boundaries prior to the fatigue crack initiation. Fatigue cracks were nucleated at the boundary between the grains with the concentrated complex slip deformation. The crystallographic orientation analysis revealed that the complex slip deformation was formed by the cross slip of active slip systems with the largest Schmid factors. It was found that the relation between the fatigue limit and grain size of the ultrafine-grained specimens does not conform to the Hall-Petch relation of conventional ferritic-pearlitic steel due to the texture and cross slip.