The introduction of conductive polymers can significantly improve the dielectric constant of polymer-based materials but results in overly large increases of dielectric loss. Herein, uniform-sized and easily dispersed polyaniline nanorods (dPANI@PVP) were synthesized by applying a dynamically interfacial polymerization method and encapsulating the nanorods with poly(vinyl pyrrolidone) (PVP). The synthesized dPANI@PVP was then used to fabricate poly(vinylidene fluoride)-based (PVDF) nanocomposites that displayed a high dielectric constant and low dielectric loss. Morphological and structural analyses showed the insulating PVP shell to have a thickness of 5–10 nm and to be wrapped on the dPANI nanorod surface, which not only provided a barrier layer between the conductive dPANI nanorods, but also facilitated an excellent dispersion of dPANI nanorods in the PVDF matrix. As a result, the dPANI@PVP/PVDF nanocomposites showed a relatively low dielectric loss while maintaining a sufficiently high dielectric constant. Specifically, when the loading of dPANI@PVP was 9.5 wt%, the dielectric constant of the nanocomposite reached 174 at 100 Hz, which is about 20 times higher than that of pure PVDF. Moreover, there was only a small increase of the dielectric loss from 0.06 for pure PVDF to 0.17 for the nanocomposite.