A number of type I JAK2 inhibitors binding to the active conformation have emerged so far, while the type II JAK2 inhibitors binding also to the allosteric site adjacent to the ATP-binding pocket in the inactive conformation, have been less studied. In this paper, the individual mechanisms and the differences of the type I and type II inhibitors binding into DFG-in and DFG-out JAK2 were characterized by molecular docking, molecular dynamic simulations, free energy calculations and decomposition. Comparison between the type I thienopyridine inhibitors suggest that the binding affinities are mainly determined by hydrogen bonds with residues E930 and L932 and strong hydrophobic contributions with V863, Y931 and R980 in the ATP-binding pocket. For type II JAK2 inhibitors BBT594 and CHZ868, not only do their type I heads bind in a similar mode in the ATP-binding site, but also the type II tail and the linker can be anchored by hydrogen bonds with E898 and D994 and lead to substantial hydrophobic contributions with residues L902, L893 and F995 in the unique allosteric site. In addition, new type II JAK2 inhibitors are reasonably designed and further evaluated by a comprehensive modeling study. The expected interactions in the ATP-binding and allosteric pocket indicate that the structural and energetic insights into JAK2 inhibitors can facilitate the process of more promising type II JAK2 inhibitors.