Motion analysis in computer vision is a well-studied problem with numerous applications. In particular, the tasks of optical flow estimation and tracking are of increasing interest. In this paper, we propose a level set approach to address both aspects of motion analysis. Our approach relies on the propagation of smooth interfaces to perform tracking while using an incremental estimation of the motion models. Implicit representations are used to represent moving objects, and capture their motion parameters. Information from different sources like a boundary attraction term, a background subtraction component and a visual consistency constraint are considered. The Euler–Lagrange equations within a gradient descent method lead to a flow that deforms a set of initial curve towards the object boundaries as well an incremental robust estimator of their apparent motion. Partial extension of the proposed framework to address dense motion estimation and the case of moving observer is also presented. Promising results demonstrate the performance of the method.