Thermoelectric (TE) devices can save energy by generating electricity from waste heat. For industrial application of TE power generation, TE materials with high conversion efficiency and that are non-toxic and inexpensive are required. The nanostructured silicon-germanium (Si-Ge) alloy is one possible candidate for such TE materials. Nanostructuring is an effective way to improve the conversion efficiency of materials because it dramatically reduces the lattice thermal conductivity (κ_lat). Here, we experimentally demonstrate effective phonon blocking without carrier mobility deterioration in bulk Si-Ge alloys containing phosphorous-rich nanoscale precipitates connected coherently or semi-coherently with the matrix phase. When the Ge content was less than 5%, the nanoscale precipitates effectively scattered heat-carrying phonons, leading to a sufficient reduction in κ_lat. However, at higher Ge content compositions, phonon scattering by Ge substitution with Si was more predominant than phonon scattering by nanoscale precipitates for the reduction of κ_lat. As a result, significant enhancement of zT was achieved at low Ge contents.