We investigate the effect of nitrogen and boron doping on Li diffusion through defected graphene using first principles based density functional theory. While a high energy barrier rules out the possibility of Li- diffusion through the pristine graphene, the barrier reduces with the incorporation of defects. Among the most common defects in pristine graphene, Li diffusion through the divacancy encounters the lowest energy barrier of 1.34 eV. The effect of nitrogen and boron doping on the Li diffusion through doped defected-graphene sheets has been studied. N-doping in graphene with a monovacancy reduces the energy barrier significantly. The barrier reduces with the increasing number of N atoms. On the other hand, for N doped graphene with a divacancy, Li binds in the plane of the sheet, with an enhanced binding energy. The B doping in graphene with a monovacancy leads to the enhancement of the barrier. However, in the case of B-doped graphene with a divacancy, the barrier reduces to 1.54 eV, which could lead to good kinetics. The barriers do not change significantly with B concentration. Therefore, divacancy, B and N doped defected graphene has emerged as a better alternative to pristine graphene as an anode material for Li ion battery.