In the present work, we have developed a novel method for in situ controllable preparation of gold nanorods (GNRs) within the thermo-responsive P(NIPAM-NVP) hydrogel networks by regulating the rates of the reduction and the mass transfer. When the pH value of the medium is gradually decreased, the reduction rate of gold ions is significantly decreased and the morphologies of the gold nanostructures in hydrogel networks are changed from gold nanoparticles to GNRs with different aspect ratios. Besides, lower cross-linking density results in the increased diffusion coefficient and reduced diffusion resistance of the solute in the hydrogels. It is an advantage for maintaining an appropriate balance between the reduction rate and the diffusion rate, in turn, improving the yield of GNRs. The resulting composites combine the thermo-responsive behavior of P(NIPAM-NVP) hydrogels with the optical properties of GNRs. Their localized surface plasmon resonance and the surface enhanced Raman scattering effects can be changed by adjusting the temperature of the hydrogels. The composites are utilized as SERS substrates and demonstrate an excellent thermo-responsive enhancement effect.