Camera orientation, calibration and inverse perspective with uncertainties: A Bayesian method applied to area estimation from diverse photographs

Abstract

Large collections of images have become readily available through modern digital catalogs, from sources as diverse as historical photographs, aerial surveys, or user-contributed pictures. Exploiting the quantitative information present in such wide-ranging collections can greatly benefit studies that follow the evolution of landscape features over decades, such as measuring areas of glaciers to study their shrinking under climate change. However, many available images were taken with low-quality lenses and unknown camera parameters. Useful quantitative data may still be extracted, but it becomes important to both account for imperfect optics, and estimate the uncertainty of the derived quantities. In this paper, we present a method to address both these goals, and apply it to the estimation of the area of a landscape feature traced as a polygon on the image of interest. The technique is based on a Bayesian formulation of the camera calibration problem. First, the probability density function (PDF) of the unknown camera parameters is determined for the image, based on matches between 2D (image) and 3D (world) points together with any available prior information. In a second step, the posterior distribution of the feature area of interest is derived from the PDF of camera parameters. In this step, we also model systematic errors arising in the polygon tracing process, as well as uncertainties in the digital elevation model. The resulting area PDF therefore accounts for most sources of uncertainty. We present validation experiments, and show that the model produces accurate and consistent results. We also demonstrate that in some cases, accounting for optical lens distortions is crucial for accurate area determination with consumer-grade lenses. The technique can be applied to many other types of quantitative features to be extracted from photographs when careful error estimation is important.

Introduction

A large amount of quantitative physical landscape information can be extracted from terrestrial, aerial and satellite imagery using various photogrammetric techniques (Streilein, 1994, Gruen and Li, 1995, Haala and Brenner, 1999, Küng et al., 2012, Feurer and Vinatier, 2018). Inverse perspective methods (e.g. monoplotting), as reviewed by Criminisi, 2001, Förstner and Wrobel, 2016, aim at extracting referenced spatial data from a single picture. Such methods have been used to extract data from either aerial, satellite or terrestrial imagery (Jordan et al., 2005, Bozzini et al., 2012, Murtiyoso et al., 2014, Produit et al., 2016). Inverse perspective methods are particularly used in the study of Earth surface processes and landscape evolution to produce or update geological and geomorphological map data (Warner et al., 1993, Jauregui et al., 2002, Micheletti et al., 2015, Scapozza et al., 2016) or, among others, in civil engineering and building stability assessment (Murtiyoso et al., 2014). The methods have also found a particular echo in the community of cryospheric sciences, as they allow to reconstruct and monitor the evolution of glaciers over different time scales, ranging from centennial to annual fluctuations (Wiesmann et al., 2012, Piermattei et al., 2015, Čekada et al., 2016), as well as further the understanding of glacier mass balance processes (Chapuis et al., 2010).

Inverse perspective methods also allow to tap into a wealth of quantitative information present in large and diverse databases of images readily accessible from the Internet, such as historical records, aerial surveys, or user-contributed pictures. However, these images are of uneven quality: many were taken without scientific intent, often with low-quality lenses and unknown camera parameters, and are sometimes only available in low resolution. These limitations can introduce significant uncertainties and biases in the information obtained from camera orientation and calibration. Useful quantitative data may still be extracted, but accounting for potential lens distortions and quantifying the uncertainty of the results become important.

In this paper, we present a method to address both goals, applying it to the estimation of the area of landscape features, with the determination of the areas of mountain glaciers in mind. The present technique is a two-step process based on Bayesian inference.

First, the unknown camera parameters, including lens optical distortions, are estimated using a Bayesian formulation of the camera orientation with calibration problem, in the form of a spatial resection problem: the posterior probability density function (PDF) of camera parameters is obtained from matches between 2D points in the image and 3D points in world coordinates, together with any available prior information. Bayesian approaches to camera calibration were presented by several authors, either based on finding a single value of the parameters which maximize a posterior distribution (e.g. Valkenburg, 1998, Zhang, 2000) or using the whole resulting posterior distribution more extensively (e.g. Sundareswara and Schrater, 2005). In our work, we keep the full statistical information contained in the posterior distribution of camera parameters, by generating samples distributed according to the posterior distribution using Markov chain Monte Carlo (MCMC) sampling.

In a second step, the posterior PDF of the feature area is derived from the posterior of camera parameters by solving an inverse perspective problem. The outline of the feature of interest is manually traced on the photograph as a polygon, which is then back projected from the 2D image onto the 3D world using a digital elevation model (DEM). This back projection step accounts for uncertainties on the camera parameters, and attempts to model possible systematic errors introduced by the polygon tracing step, together with uncertainties introduced by the DEM. In particular, we propose a model for DEM errors for which both the root mean square error (RMSE) and spatial autocorrelation scale are locally varying. The resulting PDF of the back projected 3D feature area therefore contains information about most of the uncertainties of the process.

More generally, we attempt to unify the camera orientation with calibration, uncertainty modeling and inverse perspective problems into a statistically consistent framework which can be extended to similar classes of problems and uncertainty models.

We stress that given the diverse nature and sources of our target images and the fact that many were taken using low-quality or unknown equipment, the focus of this paper is more on uncertainty estimation than on very accurate photogrammetric techniques. The reconstructed camera location, for example, cannot be expected to be more accurate than a few meters, given that we will be working with low resolution landscape images, and that our 3D ground control point coordinates will not come from precision geodetic sources.

We start by describing our Bayesian formulation of the camera orientation with calibration problem in Section 2, and our polygon back projection method in Section 3. Section 4 describes details of our implementation, including and the posterior probability density sampling process. In Section 5, we present test problems for validation, before discussing the results in Section 6 and concluding in Section 7.