Representative examples of recently developed polymers bearing either reversible or irreversible photosensitive chromophores are reviewed. By integrating these chromophores in different positions of polymers, various types of photo-responsive polymers including main-chain conjugated polymers, homopolymers, telechelic polymers, block copolymers, dendritic polymers, and supramolecular polymers, are obtained. Both their chemical and physical properties can be remotely controlled by light, leading to a wide range of applications such as controlled drug delivery, micropattern, responsive hydrogel, degradable materials, and soft actuators.