We have developed a method that can remove view-disturbing noise from image sequences by spatio-temporal image processing. In outdoor environments on rainy days, pictures taken by a camera are often degraded because of adherent noise, such as water drops on the surface of the lens-protecting glass of the camera. To solve this problem, our method uses image sequences captured with a pan-tilt camera. The method uses a spatio-temporal image to extract adherent noise regions by examining trajectory differences between adherent noise and other objects in cross-section images. Finally, adherent noise regions are filled in with surrounding information in the cross-section image. Experimental results showed our method is effective.