Skip to main content
SpringerLink
Log in

Nesting forward-mode AD in a functional framework

  • Published:
Higher-Order and Symbolic Computation

Abstract

We discuss the augmentation of a functional-programming language with a derivative-taking operator implemented with forward-mode automatic differentiation (AD). The primary technical difficulty in doing so lies in ensuring correctness in the face of nested invocation of that operator, due to the need to distinguish perturbations introduced by distinct invocations. We exhibit a series of implementations of a referentially-transparent forward-mode-AD derivative-taking operator, each of which uses a different non-referentially-transparent mechanism to distinguish perturbations. Even though the forward-mode-AD derivative-taking operator is itself referentially transparent, we hypothesize that one cannot correctly formulate this operator as a function definition in current pure dialects of Haskell.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Church, A.: The Calculi of Lambda Conversion. Princeton University Press, Princeton (1941)

    Google Scholar 

  2. Claessen, K., Sands, D.: Observable sharing for functional circuit description. In: Proc. of Asian Computer Science Conference (ASIAN). Springer, New York (1999)

    Google Scholar 

  3. Clifford, W.K.: Preliminary sketch of Bi-quaternions. Proc. Lond. Math. Soc. 4, 381–95 (1873)

    Article  Google Scholar 

  4. Flatt, M.: PLT MzScheme: language manual. Technical Report PLT-TR05-1-v300, PLT Scheme Inc. (2005)

  5. Karczmarczuk, J.: Functional differentiation of computer programs. In: Proceedings of the III ACM SIGPLAN International Conference on Functional Programming. Baltimore, MD, pp. 195–203 (1998)

  6. Karczmarczuk, J.: Lazy differential algebra and its applications. In: Workshop, III International Summer School on Advanced Functional Programming. Braga, Portugal (1998)

  7. Karczmarczuk, J.: Functional coding of differential forms. In: Scottish Workshop on FP (1999)

  8. Karczmarczuk, J.: Functional differentiation of computer programs. Higher-Order Symb. Comput. 14, 35–57 (2001)

    Article  MATH  Google Scholar 

  9. Leibnitz, G.W.: A new method for maxima and minima as well as tangents, which is impeded neither by fractional nor irrational quantities, and a remarkable type of calculus for this. Acta Eruditorum (1664)

  10. Newton, I.: De quadratura curvarum. In Optiks, 1704 edition. Appendix

  11. Nilsson, H.: Functional automatic differentiation with Dirac impulses. In: Proceedings of the Eighth ACM SIGPLAN International Conference on Functional Programming. Uppsala, Sweden, pp. 153–164. ACM Press, New York (2003)

    Chapter  Google Scholar 

  12. Pearlmutter, B.A., Siskind, J.M.: Lazy multivariate higher-order forward- mode AD. In: Proceedings of the 2007 Symposium on Principles of Programming Languages. Nice, France, pp. 155–160 (2007)

  13. Sussman, G.J., Wisdom, J., Mayer, M.E.: Structure and Interpretation of Classical Mechanics. MIT Press, Cambridge (2001)

    MATH  Google Scholar 

  14. Wengert, R.E.: A simple automatic derivative evaluation program. Commun. ACM 7(8), 463–464 (1964)

    Article  MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Electrical and Computer Engineering, Purdue University, West Lafayette, USA

    Jeffrey Mark Siskind

  2. Hamilton Institute, NUI Maynooth, Maynooth, Co. Kildare, Ireland

    Barak A. Pearlmutter

Authors
  1. Jeffrey Mark Siskind
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Barak A. Pearlmutter
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Barak A. Pearlmutter.

Additional information

Supported, in part, by US NSF grant CCF-0438806, Science Foundation Ireland grant 00/PI.1/C067, and the Higher Education Authority of Ireland (An tÚdarás Um Ard-Oideachas). Any opinions, findings, and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the supporting agencies.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Siskind, J.M., Pearlmutter, B.A. Nesting forward-mode AD in a functional framework. Higher-Order Symb Comput 21, 361–376 (2008). https://doi.org/10.1007/s10990-008-9037-1

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10990-008-9037-1

Keywords

Search

Navigation