Abstract
Recent approaches for variational motion estimation typically either rely on first or second order regularisation strategies. While first order strategies are more appropriate for scenes with fronto-parallel motion, second order constraints are superior if it comes to the estimation of affine flow fields. Since using the wrong regularisation order may lead to a significant deterioration of the results, it is surprising that there has not been much effort in the literature so far to determine this order automatically. In our work, we address the aforementioned problem in two ways. (i) First, we discuss two anisotropic smoothness terms of first and second order, respectively, that share important structural properties and that are thus particularly suited for being combined within an order-adaptive variational framework. (ii) Secondly, based on these two smoothness terms, we develop four different variational methods and with it four different strategies for adaptively selecting the regularisation order: a global and a local strategy based on half-quadratic regularisation, a non-local approach that relies on neighbourhood information, and a region based method using level sets. Experiments on recent benchmarks show the benefits of each of the strategies. Moreover, they demonstrate that adaptively combining different regularisation orders not only allows to outperform single-order strategies but also to obtain advantages beyond the ones of a frame-wise selection.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Amiaz, T., Kiryati, N.: Piecewise-smooth dense optical flow via level sets. Int. J. Comput. Vis. 68(2), 111–124 (2006)
Baker, S., Scharstein, D., Lewis, J.P., Roth, S., Black, M.J., Szeliski, R.: A database and evaluation methodology for optical flow. Int. J. Comput. Vis. 92(1), 1–31 (2011)
Black, M.J., Anandan, P.: The robust estimation of multiple motions: parametric and piecewise-smooth flow fields. Comput. Vis. Image Underst. 63(1), 75–104 (1996)
Blake, A., Zisserman, A.: Visual Reconstruction. MIT Press, Cambridge (1987)
Braux-Zin, J., Dupont, R., Bartoli, A.: A general dense image matching framework combining direct and feature-based costs. In: Proceedings of the International Conference on Computer Vision, pp. 185–192 (2013)
Bredies, K., Kunisch, K., Pock, T.: Total generalized variation. SIAM J. Imaging Sci. 3(3), 492–526 (2010)
Brox, T., Bruhn, A., Papenberg, N., Weickert, J.: High accuracy optical flow estimation based on a theory for warping. In: Pajdla, T., Matas, J. (eds.) ECCV 2004. LNCS, vol. 3024, pp. 25–36. Springer, Heidelberg (2004). doi:10.1007/978-3-540-24673-2_3
Butler, D.J., Wulff, J., Stanley, G.B., Black, M.J.: A naturalistic open source movie for optical flow evaluation. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds.) ECCV 2012. LNCS, vol. 7577, pp. 611–625. Springer, Heidelberg (2012). doi:10.1007/978-3-642-33783-3_44
Bruhn, A., Weickert, J.: Towards ultimate motion estimation: combining highest accuracy with real-time performance. In: Proceedings of the International Conference on Computer Vision, pp. 749–755 (2005)
Charbonnier, P., Blanc-Féraud, L., Aubert, G., Barlaud, M.: Deterministic edge-preserving regularization in computed imaging. IEEE Trans. Image Process. 6(2), 298–311 (1997)
Cremers, D., Soatto, S.: Motion competition: a variational approach to piecewise parametric motion segmentation. Int. J. Comput. Vis. 62(3), 249–265 (2005)
Demetz, O., Stoll, M., Volz, S., Weickert, J., Bruhn, A.: Learning brightness transfer functions for the joint recovery of illumination changes and optical flow. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8689, pp. 455–471. Springer, Cham (2014). doi:10.1007/978-3-319-10590-1_30
Förstner, W. Gülch., E.: A fast operator for detection and precise location of distinct points, corners and centres of circular features. In: Proceedings of the ISPRS Intercommission Conference on Fast Processing of Photogrammetric Data, pp. 281–305 (1987)
Geiger, A., Lenz, P., Urtasun, R.: Are we ready for autonomous driving? The KITTI vision benchmark suite. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3354–3361 (2012)
Hafner, D., Schroers, C., Weickert, J.: Introducing maximal anisotropy into second order coupling models. In: Gall, J., Gehler, P., Leibe, B. (eds.) GCPR 2015. LNCS, vol. 9358, pp. 79–90. Springer, Cham (2015). doi:10.1007/978-3-319-24947-6_7
Horn, B., Schunck, B.: Determining optical flow. Artif. Intell. 17, 185–203 (1981)
Kuschk, G., Cremers, D.: Fast and accurate large-scale stereo reconstruction using variational methods. In: Proceedings of the ICCV Workshops, pp. 1–8 (2013)
Lenzen, F., Becker, F., Lellmann, J.: Adaptive second-order total variation: an approach aware of slope discontinuities. In: Kuijper, A., Bredies, K., Pock, T., Bischof, H. (eds.) SSVM 2013. LNCS, vol. 7893, pp. 61–73. Springer, Heidelberg (2013). doi:10.1007/978-3-642-38267-3_6
Menze, M., Geiger, A.: Object scene flow for autonomous vehicles. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3061–3070 (2015)
Nagel, H.-H., Enkelmann, W.: An investigation of smoothness constraints for the estimation of displacement vector fields from image sequences. IEEE Trans. Pattern Anal. Mach. Intell. 8(5), 565–593 (1986)
Osher, S., Sethian, J.A.: Fronts propagating with curvature-dependent speed: algorithms based on Hamilton-Jacobi formulations. J. Comput. Phys. 79(1), 12–49 (1988)
Perona, P., Malik, J.: Scale space and edge detection using anisotropic diffusion. IEEE Trans. Pattern Anal. Mach. Intell. 2, 629–639 (1990)
Ranftl, R., Bredies, K., Pock, T.: Non-local total generalized variation for optical flow estimation. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8689, pp. 439–454. Springer, Cham (2014). doi:10.1007/978-3-319-10590-1_29
Vogel, C., Roth, S., Schindler, K.: An evaluation of data costs for optical flow. In: Weickert, J., Hein, M., Schiele, B. (eds.) GCPR 2013. LNCS, vol. 8142, pp. 343–353. Springer, Heidelberg (2013). doi:10.1007/978-3-642-40602-7_37
Volz, S., Bruhn, A., Valgaerts, L., Zimmer, H.: Modeling temporal coherence for optical flow. In: Proceedings of the International Conference on Computer Vision (2011)
Weickert, J., Schnörr, C.: A theoretical framework for convex regularizers in PDE-based computation of image motion. Int. J. Comput. Vis. 45(3), 245–264 (2001)
Xu, L., Jia, J., Matsushita, Y.: Motion detail preserving optical flow estimation. IEEE Trans. Pattern Anal. Mach. Intell. 34, 1744–1757 (2012)
Zimmer, H., Bruhn, A., Weickert, J.: Optic flow in harmony. Int. J. Comput. Vis. 93(3), 368–388 (2011)
Acknowledgements
We thank the German Research Foundation (DFG) for financial support within project B04 of SFB/Transregio 161.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2017 Springer International Publishing AG
About this paper
Cite this paper
Maurer, D., Stoll, M., Bruhn, A. (2017). Order-Adaptive Regularisation for Variational Optical Flow: Global, Local and in Between. In: Lauze, F., Dong, Y., Dahl, A. (eds) Scale Space and Variational Methods in Computer Vision. SSVM 2017. Lecture Notes in Computer Science(), vol 10302. Springer, Cham. https://doi.org/10.1007/978-3-319-58771-4_44
Download citation
DOI: https://doi.org/10.1007/978-3-319-58771-4_44
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-58770-7
Online ISBN: 978-3-319-58771-4
eBook Packages: Computer ScienceComputer Science (R0)