The instability of a bed of particles sheared by a viscous fluid is investigated theoretically. The viscous flow over the wavy bed is first calculated, and the bed shear stress is derived. The particle transport rate induced by this bed shear stress is calculated from the viscous resuspension theory of Leighton & Acrivos (1986). Mass conservation of the particles then gives explicit expressions for the wave velocity and growth rate, which depend on four dimensionless parameters: the wavenumber, the fluid thickness, a viscous length and the shear stress. The mechanism of the instability is given. It appears that for high enough fluid-layer thickness, long-wave instability arises as soon as grains move, while short waves are stabilized by gravity. For smaller fluid thickness, the destabilizing effect of fluid inertia is reduced, so that the moving at bed is stable for small shear stress, and unstable for high shear stress. The most amplified wavelength scales with the viscous length, in agreement with the few available experiments for small particle Reynolds numbers. The results are also compared with related studies for turbulent flow.