This paper describes a method for motion-compensated compression of video sequences. The proposed method enables very efficient encoding of video frames partitioned into arbitrary-shaped regions. The implementation described in this paper employs simple quadtree-based segmentation of video frames and a polynomial model of the motion vector field. Polynomial motion vector fields are estimated using an iterative minimization method. Application of complex polynomial motion vector fields to coding of video requires their compact encoding, especially for low bit-rate applications. The paper proposes novel algorithms enabling low-complexity encoding of polynomial motion vector fields which are optimal in least mean square error sense. Specifically, an efficient method for merging of regions resulting from quadtree segmentation is presented. The advantage of this method is its ability to achieve large reduction in the number of regions in typical coded sequences with only a minor increase of prediction error. An algorithm for reduction of the number of polynomial motion vector field coefficients is also presented. This algorithm enables adaptation of the motion vector field model to the complexity of motion in the coded scene. The problem of quantization of polynomial motion coefficients is also addressed. Performance of the proposed algorithms has been evaluated in a simple motion-compensated DCT codec and compared to an ITU-T H.263 codec. Simulations show that a consistent reduction of bit-rate in excess of 25% can be achieved over a wide range of sequences. Depending on the envisaged application the described algorithms can be used together with segmentation of desired accuracy and therefore provide simultaneously high coding efficiency as well as ability of content-based encoding of video.