This paper proposes a method for matching images based on their higher order moments without knowing the point correspondences. It is assumed that the disparity between the images can be explained by an affine transformation. The second-order statistics is used to transform the image points into canonical form, which reduces the affine matching problem for determining an orthonormal transformation matrix between the two point sets. Next, higher order complex moments are used to solve the remaining transformation. These affine moment descriptors are expressed in terms of the central moments of the original data. It is also shown that the resulting descriptors can be converted into affine moment invariants. A general framework for deriving affine moment descriptors as well as moment invariants is described. The experiments carried out with simulated data and real images indicate that the proposed method utilizing the second- and third-order statistics can provide good alignment results from noisy and spurious observations.