Skip to main content
SpringerLink
Log in

Development Efforts for Reproducible Research: Platform, Library and Editorial Investment

  • Conference paper
  • First Online:
Reproducible Research in Pattern Recognition (RRPR 2022)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 14068))

Included in the following conference series:

  • 71 Accesses

Abstract

Reproducible research in pattern recognition can be viewed from a number of angles, including code execution, platforms that promote reproducibility, code sharing, or the release of libraries providing access to relevant algorithms in the corresponding disciplines. In this work, after recalling the motivation and classic definitions of reproducible research, we propose an updated overview of the main platforms that might be used for reproducible research. We then review the different libraries that are commonly used by the pattern recognition, computer vision, imaging and geometry processing communities, and we share our experience of developing a research library. In the third part, new advanced editorial investments will be presented, such as the IPOL journal or other IPOL-inspired new initiatives like OVD-SaaS.

This research was made possible by support from the French National Research Agency, in the framework of the projects WoodSeer, ANR-19-CE10-011, ULTRA-LEARN, ANR-20-CE23-0019, and by the SESAME’s OVD-SaaS project from Région Île de France and BPI France, and Ministry of Science, Technology and Innovation of Colombia (Minciencias), call 885 of 2020.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    Data extracted from https://portal.paperswithcode.com on 15 May 2023.

  2. 2.

    https://emscripten.org/.

  3. 3.

    https://github.com/xtensor-stack/xtensor.

  4. 4.

    https://github.com/catchorg/Catch2.

  5. 5.

    https://readthedocs.org/.

  6. 6.

    https://pypi.org/.

  7. 7.

    https://about.codecov.io/.

  8. 8.

    https://www.softwareheritage.org/2020/06/11/ipol-and-swh/?lang=es.

References

  1. Donoho, D.L., Maleki, A., Ur Rahman, I., Shahram, M., Stodden, V.: Reproducible research in computational harmonic analysis. Comput. Sci. Eng. 11(1), 8–18 (2009)

    Article  Google Scholar 

  2. Colom, M., Kerautret, B., Krähenbühl, A.: An overview of platforms for reproducible research and augmented publications. In: Kerautret, B., Colom, M., Lopresti, D., Monasse, P., Talbot, H. (eds.) RRPR 2018. LNCS, vol. 11455, pp. 25–39. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-23987-9_2

    Chapter  Google Scholar 

  3. Yildiz, B., et al.: ReproducedPapers.org: openly teaching and structuring machine learning reproducibility. In: Kerautret, B., Colom, M., Krähenbühl, A., Lopresti, D., Monasse, P., Talbot, H. (eds.) RRPR 2021. LNCS, vol. 12636, pp. 3–11. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-76423-4_1

    Chapter  Google Scholar 

  4. Lucic, A., Bleeker, M., Jullien, S., Bhargav, S., de Rijke, M.: Reproducibility as a mechanism for teaching fairness, accountability, confidentiality, and transparency in artificial intelligence (2022)

    Google Scholar 

  5. Rampin, R., Chirigati, F., Steeves, V., Freire, J.: ReproServer: making reproducibility easier and less intensive (2018). https://arxiv.org/abs/1808.01406

  6. Rampin, R., Chirigati, F., Shasha, D., Freire, J., Steeves, V.: ReproZip: the reproducibility packer. J. Open Source Softw. 1(8), 107 (2016)

    Article  Google Scholar 

  7. Šimko, T., Heinrich, L., Hirvonsalo, H., Kousidis, D., Rodríguez, D.: REANA: a system for reusable research data analyses. In: EPJ web of conferences, vol. 214, p. 06034. EDP Sciences (2019)

    Google Scholar 

  8. Bonneel, N., Coeurjolly, D., Digne, J., Mellado, N.: Code replicability in computer graphics. ACM Trans. Graph. 39(4), 93-1 (2020)

    Google Scholar 

  9. Stojnic, R., Taylor, R.: Papers with code-a facebook AI project (2018). https://paperswithcode.com. Accessed 30 Aug 2022

  10. Bradski, G.: The OpenCV library. Dr. Dobb’s J. Softw. Tools (2000)

    Google Scholar 

  11. McCormick, M., Liu, X., Jomier, J., Marion, C., Ibanez, L.: ITK: enabling reproducible research and open science. Front. Neuroinf. 8, 13 (2014)

    Article  Google Scholar 

  12. Rusu, R.B., Cousins, S.: 3D is here: point cloud library (PCL). In: IEEE International Conference on Robotics and Automation (ICRA). IEEE (2011)

    Google Scholar 

  13. The CGAL Project. CGAL User and Reference Manual. 5.4.1 edition (2022)

    Google Scholar 

  14. Tschumperlé, D.: The CIMG library. In: IPOL 2012 Meeting on Image Processing Libraries, p. 4 (2012)

    Google Scholar 

  15. Geogram: a programming library with geometric algorithms. https://github.com/BrunoLevy/geogram

  16. Roynard, M., Carlinet, E., Géraud, T.: An image processing library in modern C++: getting simplicity and efficiency with generic programming. In: Kerautret, B., Colom, M., Lopresti, D., Monasse, P., Talbot, H. (eds.) RRPR 2018. LNCS, vol. 11455, pp. 121–137. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-23987-9_12

    Chapter  Google Scholar 

  17. Auber, D.: Tulip-a huge graph visualization framework. In: Graph Drawing Software, pp. 105–126. Springer, Heidelberg (2004). https://doi.org/10.1007/978-3-642-18638-7_5

  18. Vigra: Vision with generic algorithms. https://ukoethe.github.io/vigra. Accessed May 2022

  19. Dgtal: Digital geometry tools and algorithms library. http://dgtal.org

  20. Moulon, P., Monasse, P., Perrot, R., Marlet, R.: OpenMVG: open multiple view geometry. In: Kerautret, B., Colom, M., Monasse, P. (eds.) RRPR 2016. LNCS, vol. 10214, pp. 60–74. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-56414-2_5

    Chapter  Google Scholar 

  21. Tierny, J., Favelier, G., Levine, J.A., Gueunet, C., Michaux, M.: The topology ToolKit. IEEE Trans. Vis. Comput. Graph. (2017). https://topology-tool-kit.github.io/

  22. Perret, B., Chierchia, G., Cousty, J., Guimarães, S.J. F., Kenmochi, Y., Najman, L.: Higra: hierarchical graph analysis. SoftwareX 10, 1–6 (2019). https://github.com/higra/Higra

  23. Harris, C.R., et al.: Array programming with NumPy. Nature 585(7825), 357–362 (2020)

    Article  Google Scholar 

  24. Lam, S.K., Pitrou, A., Seibert, S.: Numba: a llvm-based python jit compiler. In: Proceedings of the Second Workshop on the LLVM Compiler Infrastructure in HPC, pp. 1–6 (2015)

    Google Scholar 

  25. Jakob, W., Rhinelander, J., Moldovan, D.: pybind11—seamless operability between c++11 and python (2016). https://github.com/pybind/pybind11

  26. Nicolaï, A., et al.: The approach to reproducible research of the image processing on line (ipol) journal. Informatio 27(1), 76–112 (2022)

    Google Scholar 

  27. Colom, M., Dagobert, T., Franchis, C.D., Gioi, R.G.V., Hessel, C., Morel, J.M.: Using the ipol journal for online reproducible research in remote sensing. IEEE J. Sel. Topics Appl. Earth Obs. Remote Sens. 13, 6384–6390 (2020)

    Google Scholar 

  28. Baron, A.-F., Boulant, O., Panico, I., Vayatis, N.: A compartmental epidemiological model applied to the Covid-19 epidemic. Image Process. Line 11, 105–119 (2021). https://doi.org/10.5201/ipol.2021.323

    Article  MathSciNet  Google Scholar 

  29. Di Cosmo, R., Zacchiroli, S.: Software heritage: why and how to preserve software source code. In: iPRES 2017–14th International Conference on Digital Preservation, pp. 1–10 (2017)

    Google Scholar 

Download references

Acknowledgement

The authors would like to thank Burak Yildiz from Delft University of Technology for providing statistics on reproducedpapers.org platform and Dean Pleban from the Dagshub platform for helping and orienting the authors to measure user activity. They also thank the reviewers for their valuable comments and corrections and Bruno Levy for pointing us the usage statistics of the Geogram Library.

Author information

Authors and Affiliations

  1. Université Paris-Saclay, ENS Paris-Saclay, CNRS, Centre Borelli, 91190, Gif-sur-Yvette, France

    Miguel Colom & José Armando Hernández

  2. Univ Lyon, Univ Lyon 2, CNRS, INSA Lyon, UCBL, LIRIS, UMR5205, F-69676, Bron, France

    Bertrand Kerautret

  3. LIGM, Univ Gustave Eiffel, CNRS, ESIEE Paris, 77454, Marne-la-Vallee, France

    Benjamin Perret

Authors
  1. Miguel Colom
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. José Armando Hernández
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Bertrand Kerautret
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Benjamin Perret
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Bertrand Kerautret .

Editor information

Editors and Affiliations

  1. Lumière University Lyon 2, Bron, France

    Bertrand Kerautret

  2. Ecole Normale Superieure Paris, Gif-sur-Yvette, France

    Miguel Colom

  3. University of Stasbourg, Illkirch Cedex, France

    Adrien Krähenbühl

  4. Lehigh University, Bethlehem, PA, USA

    Daniel Lopresti

  5. Ecole des Ponts Paris Tech, Marne- la- Vallée Cedex 2, France

    Pascal Monasse

  6. ESIEE Paris, Université Gustave Eiffel, Noisy le Grand Cedex, France

    Benjamin Perret

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Colom, M., Hernández, J.A., Kerautret, B., Perret, B. (2023). Development Efforts for Reproducible Research: Platform, Library and Editorial Investment. In: Kerautret, B., Colom, M., Krähenbühl, A., Lopresti, D., Monasse, P., Perret, B. (eds) Reproducible Research in Pattern Recognition. RRPR 2022. Lecture Notes in Computer Science, vol 14068. Springer, Cham. https://doi.org/10.1007/978-3-031-40773-4_1

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-40773-4_1

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-40772-7

  • Online ISBN: 978-3-031-40773-4

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Societies and partnerships

Search

Navigation