research-article

Approximating Inverse Cumulative Distribution Functions to Produce Approximate Random Variables

Authors:
Michael Giles

Mathematical Institute, Oxford University, UK

Mathematical Institute, Oxford University, UK

0000-0002-5445-3721
View Profile

,
Oliver Sheridan-Methven

Mathematical Institute, Oxford University, UK

Mathematical Institute, Oxford University, UK

0000-0003-1088-7138
View Profile

Authors Info & Claims

ACM Transactions on Mathematical Software Volume 49 Issue 3Article No.: 26pp 1–29https://doi.org/10.1145/3604935

Published:19 September 2023Publication History

ACM Transactions on Mathematical Software

Abstract

For random variables produced through the inverse transform method, approximate random variables are introduced, which are produced using approximations to a distribution’s inverse cumulative distribution function. These approximations are designed to be computationally inexpensive and much cheaper than library functions, which are exact to within machine precision and, thus, highly suitable for use in Monte Carlo simulations. The approximation errors they introduce can then be eliminated through use of the multilevel Monte Carlo method. Two approximations are presented for the Gaussian distribution: a piecewise constant on equally spaced intervals and a piecewise linear using geometrically decaying intervals. The errors of the approximations are bounded and the convergence demonstrated, and the computational savings are measured for C and C++ implementations. Implementations tailored for Intel and Arm hardware are inspected alongside hardware agnostic implementations built using OpenMP. The savings are incorporated into a nested multilevel Monte Carlo framework with the Euler-Maruyama scheme to exploit the speedups without losing accuracy, offering speed ups by a factor of 5–7. These ideas are empirically extended to the Milstein scheme and the non-central χ² distribution for the Cox-Ingersoll-Ross process, offering speedups of a factor of 250 or more.

REFERENCES

[1] Abdel-Aty S. H.. 1954. Approximate formulae for the percentage points and the probability integral of the non-central \(\chi ^2\) distribution. Biometrika 41, 3/4 (1954), 538–540.Google ScholarCross Ref
[2] Abramowitz Milton and Stegun Irene A.. 1948. Handbook of Mathematical Functions with Formulas, Graphs, and Mathematical Tables. Vol. 55. US Government Printing Office. (6th printing, November 1967).Google Scholar
[3] Agrawal Nikhar, Bikineev Anton, Bristow Paul A., Guazzone Marco, Kormanyos Christopher, Holin Hubert, Lalande Bruno, Maddock John, Murphy Jeremy, Pulver Matthew, Råde Johan, Sewani Gautam, Sobotta Benjamin, Thompson Nicholas, Berg Thijs van den, Walker Daryle, Zhang Xiaogang, et al. 2020. The Boost C++ library. https://www.boost.org/. Version 1.74.0.Google Scholar
[4] Alfonsi Aurélien. 2005. On the discretization schemes for the CIR (and Bessel squared) processes. Monte Carlo Methods and Applications 11, 4 (2005), 355–384.Google ScholarCross Ref
[5] Alfonsi Aurélien. 2008. A second-order discretization scheme for the CIR process: Application to the Heston model. Preprint CERMICS hal-00143723 14 (2008).Google Scholar
[6] Alfonsi Aurélien. 2010. High order discretization schemes for the CIR process: Application to affine term structure and Heston models. Mathematics of Computation 79, 269 (2010), 209–237.Google ScholarCross Ref
[7] Asmussen Søren and Glynn Peter W.. 2007. Stochastic Simulation: Algorithms and Analysis. Vol. 57. Springer Science & Business Media.Google ScholarCross Ref
[8] Beasley J. D. and Springer S. G.. 1977. Algorithm AS 111: The percentage points of the normal distribution. Journal of the Royal Statistical Society. Series C (Applied Statistics) 26, 1 (March 1977), 118–121.Google Scholar
[9] Berkaoui Abdel, Bossy Mireille, and Diop Awa. 2008. Euler scheme for SDEs with non-Lipschitz diffusion coefficient: Strong convergence. ESAIM: Probability and Statistics 12 (2008), 1–11.Google ScholarCross Ref
[10] Best D. J. and Roberts D. E.. 1975. Algorithm AS 91: The percentage points of the \(\chi ^2\) distribution. Journal of the Royal Statistical Society. Series C (Applied Statistics) 24, 3 (1975), 385–388.Google Scholar
[11] Black Fischer and Scholes Myron. 1973. The pricing of options and corporate liabilities. Journal of Political Economy 81, 3 (1973), 637–654.Google ScholarCross Ref
[12] Box George E. P. and Muller Mervin E.. 1958. A note on the generation of random normal deviates. The Annals of Mathematical Statistics 29 (1958), 610–611.Google ScholarCross Ref
[13] Broadie Mark and Kaya Özgür. 2006. Exact simulation of stochastic volatility and other affine jump diffusion processes. Operations Research 54, 2 (2006), 217–231.Google ScholarDigital Library
[14] Brown Barry W., Lovato James, and Russell Kathy. 1994. CDFLIB: library of Fortran routines for cumulative distribution functions, inverses, and other parameters. https://people.sc.fsu.edu/jburkardt/f_src/cdflib/cdflib.html.Google Scholar
[15] Brugger Christian, Schryver Christian de, Wehn Norbert, Omland Steffen, Hefter Mario, Ritter Klaus, Kostiuk Anton, and Korn Ralf. 2014. Mixed precision multilevel Monte Carlo on hybrid computing systems. In 2014 IEEE Conference on Computational Intelligence for Financial Engineering & Economics (CIFEr). IEEE, 215–222.Google ScholarCross Ref
[16] Burkardt John. 2019. C source codes for CDFLIB. https://people.sc.fsu.edu/∼jburkardt/c_src/cdflib/cdflib.html. Accessed Wednesday 2 September 2020.Google Scholar
[17] Burkardt John. 2020. MATLAB source codes for ASA241 and ASA111. https://people.sc.fsu.edu/∼jburkardt/m_src/m_src.html. Accessed Thursday 18 June 2020.Google Scholar
[18] Bus Jacques C. P. and Dekker Theodorus Jozef. 1975. Two efficient algorithms with guaranteed convergence for finding a zero of a function. ACM Transactions on Mathematical Software (TOMS) 1, 4 (1975), 330–345.Google ScholarDigital Library
[19] Cheung Ray C. C., Lee Dong-U, Luk Wayne, and Villasenor John D.. 2007. Hardware generation of arbitrary random number distributions from uniform distributions via the inversion method. IEEE Transactions on Very Large Scale Integration (VLSI) Systems 15, 8 (2007), 952–962.Google ScholarDigital Library
[20] Cox John C., Jr Jonathan E. Ingersoll, and Ross Stephen A.. 1985. A theory of the term structure of interest rates. Econometrica 53, 2 (March 1985), 385–408–164.Google ScholarCross Ref
[21] Cozma Andrei and Reisinger Christoph. 2018. Strong order 1/2 convergence of full truncation Euler approximations to the Cox–Ingersoll–Ross process. IMA Journal of Numerical Analysis 40, 1 (October 2018), 358–376.Google ScholarCross Ref
[22] Deelstra Griselda and Delbaen Freddy. 1998. Convergence of discretized stochastic (interest rate) processes with stochastic drift term. Applied Stochastic Models and Data Analysis 14, 1 (1998), 77–84.Google ScholarCross Ref
[23] Dereich Steffen, Neuenkirch Andreas, and Szpruch Lukasz. 2012. An Euler-type method for the strong approximation of the Cox-Ingersoll-Ross process. Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences 468, 2140 (2012), 1105–1115.Google ScholarCross Ref
[24] Feller William. 1951. Two singular diffusion problems. Annals of Mathematics (1951), 173–182.Google ScholarCross Ref
[25] Fog Agner. 2018. Instruction tables: lists of instruction latencies, throughputs and micro-operation breakdowns for Intel, AMD and VIA CPUs. https://www.agner.org/optimize/instruction_tables.pdf. Updated 9 April 2018, Copenhagen University College of Engineering.Google Scholar
[26] Galassi Mark, Davies Jim, Theiler James, Gough Brian, Jungman Gerard, Alken Patrick, Booth Michael, Rossi Fabrice, Ulerich Rhys, et al. 2017. GNU Scientific Library 2.4.Google Scholar
[27] Giles Michael B.. 2008. Multilevel Monte Carlo path simulation. Operations Research 56, 3 (2008), 607–617.Google ScholarDigital Library
[28] Giles Michael B.. 2011. Approximating the erfinv function. In GPU Computing Gems, Jade Edition. Vol. 2. Elsevier, 109–116.Google Scholar
[29] Giles Michael B., Hefter Mario, Mayer Lukas, and Ritter Klaus. 2019. Random bit multilevel algorithms for stochastic differential equations. Journal of Complexity (2019).Google ScholarDigital Library
[30] Giles Michael B., Hefter Mario, Mayer Lukas, and Ritter Klaus. 2019. Random bit quadrature and approximation of distributions on Hilbert spaces. Foundations of Computational Mathematics 19, 1 (2019), 205–238.Google ScholarDigital Library
[31] Giles Michael B. and Sheridan-Methven Oliver. 2022. Analysis of nested multilevel Monte Carlo using approximate normal random variables. SIAM/ASA Journal on Uncertainty Quantification 10, 1 (2022), 200–226.Google ScholarCross Ref
[32] Giles Michael B. and Waterhouse Benjamin J.. 2009. Multilevel quasi-Monte Carlo path simulation. Advanced Financial Modelling, Radon Series on Computational and Applied Mathematics 8 (2009), 165–181.Google Scholar
[33] Glasserman Paul. 2013. Monte Carlo Methods in Financial Engineering (1st ed.). Stochastic modelling and applied probability, Vol. 53. Springer Science & Business Media.Google Scholar
[34] Gyöngy István. 1998. A note on Euler’s approximations. Potential Analysis 8, 3 (1998), 205–216.Google ScholarCross Ref
[35] Gyöngy István and Rásonyi Miklós. 2011. A note on Euler approximations for SDEs with Hölder continuous diffusion coefficients. Stochastic Processes and Their Applications 121, 10 (2011), 2189–2200.Google ScholarCross Ref
[36] Jr Cecil Hastings, Wayward Jeanne T., and Jr James P. Wong. 1955. Approximations for Digital Computers. Princeton University Press.Google Scholar
[37] Heston Steven L.. 1993. A closed-form solution for options with stochastic volatility with applications to bond and currency options. The Review of Financial Studies 6, 2 (1993), 327–343.Google ScholarCross Ref
[38] Higham Desmond J., Mao Xuerong, and Stuart Andrew M.. 2002. Strong convergence of Euler-type methods for nonlinear stochastic differential equations. SIAM Journal on Numerical Analysis 40, 3 (2002), 1041–1063.Google ScholarDigital Library
[39] Hoaglin David C.. 1977. Direct approximations for chi-squared percentage points. Journal of the American Statistical Association 72, 359 (1977), 508–515.Google ScholarCross Ref
[40] IEEE. 2008. IEEE standard for binary floating-point arithmetic. DOI: Computer Society Standards Committee. Working Group of the Microprocessor Standards Subcommittee.Google ScholarCross Ref
[41] (ISO) International Organization for Standardization. 2012. ISO/IEC 9899:2011: programming languages–C. ISO Working Group 14 (2012).Google Scholar
[42] Iserles Arieh. 1996. A First Course in the Numerical Analysis of Differential Equations. Cambridge University Press.Google ScholarDigital Library
[43] Johnson Norman L., Kotz Samuel, and Balakrishnan Narayanaswamy. 1995. Continuous Univariate Distributions. John Wiley & Sons, Ltd.Google Scholar
[44] Joy Corwin, Boyle Phelim P., and Tan Ken Seng. 1996. Quasi-Monte Carlo methods in numerical finance. Management Science 42, 6 (1996), 926–938.Google ScholarDigital Library
[45] Kloeden Peter E. and Platen Eckhard. 1999. Numerical Solution of Stochastic Differential Equations. Stochastic modelling and applied probability, Vol. 23. Springer. Corrected 3rd printing.Google Scholar
[46] Lord Gabriel J., Powell Catherine E., and Shardlow Tony. 2014. An Introduction to Computational Stochastic PDEs. Cambridge University Press.Google ScholarCross Ref
[47] Lord Roger, Koekkoek Remmert, and Dijk Dick van. 2010. A comparison of biased simulation schemes for stochastic volatility models. Quantitative Finance 10, 2 (2010), 177–194.Google ScholarCross Ref
[48] L’Ecuyer Pierre. 2016. Randomized quasi-Monte Carlo: An introduction for practitioners. In International Conference on Monte Carlo and quasi-Monte Carlo Methods in Scientific Computing. Springer, 29–52.Google Scholar
[49] Marsaglia George and Bray T. A.. 1964. A convenient method for generating normal variables. SIAM Rev. 6, 3 (1964), 260–264.Google ScholarDigital Library
[50] Marsaglia George and Tsang Wai Wan. 2000. The Ziggurat method for generating random variables. Journal of Statistical Software 5, 1 (2000), 1–7.Google Scholar
[51] Marsaglia George, Zaman Arif, and Marsaglia John C. W.. 1994. Rapid evaluation of the inverse of the normal distribution function. Statistics & Probability Letters 19, 4 (15 March 1994), 259–266.Google ScholarCross Ref
[52] MathWorks MATLAB. 2018. Statistics and Machine Learning Toolbox: User’s Guide. R2018b.Google Scholar
[53] Moshier Stephen L.. 1992. Cephes mathematical library. http://www.netlib.org/cephes/. Accessed Tuesday 8 September 2020.Google Scholar
[54] Müller Eike H., Scheichl Rob, and Shardlow Tony. 2015. Improving multilevel Monte Carlo for stochastic differential equations with application to the Langevin equation. Proceedings of the Royal Society of London A: Mathematical, Physical and Engineering Sciences 471, 2176 (2015).Google Scholar
[55] Munk Claus. 2011. Fixed Income Modelling. Oxford University Press.Google ScholarCross Ref
[56] NAG. 2017. NAG Library Manual, Mark 26. https://www.nag.com/numeric/nl/nagdoc_26/.Google Scholar
[57] Odeh Robert E. and Evans J. O.. 1974. Algorithm AS 70: The percentage points of the normal distribution. Journal of the Royal Statistical Society. Series C (Applied Statistics) 23, 1 (1974), 96–97.Google Scholar
[58] Omland Steffen, Hefter Mario, Ritter Klaus, Brugger Christian, Schryver Christian De, Wehn Norbert, and Kostiuk Anton. 2015. Exploiting mixed-precision arithmetics in a multilevel Monte Carlo approach on FPGAs. In FPGA Based Accelerators for Financial Applications. Springer, 191–220.Google ScholarCross Ref
[59] Petrogalli Francesco. 2016. A sneak peek into SVE and VLA programming. White paper.Google Scholar
[60] Powell Michael James David. 1981. Approximation Theory and Methods. Cambridge University Press.Google ScholarCross Ref
[61] Rice Stephen O.. 1968. Uniform asymptotic expansions for saddle point integrals—application to a probability distribution occurring in noise theory. Bell System Technical Journal 47, 9 (1968), 1971–2013.Google ScholarCross Ref
[62] Sankaran Munuswamy. 1959. On the non-central chi-square distribution. Biometrika 46, 1/2 (1959), 235–237.Google ScholarCross Ref
[63] Seydel Rüdiger. 2006. Tools for Computational Finance. Vol. 3. Springer.Google Scholar
[64] Shea B. L.. 1991. Algorithm AS R85: A remark on AS 91: The percentage points of the \(\chi ^2\) distribution. Journal of the Royal Statistical Society. Series C (Applied Statistics) 40, 1 (1991), 233–235.Google Scholar
[65] Sheridan-Methven Oliver. 2020. Approximating inverse cumulative distribution functions. https://github.com/oliversheridanmethven/approximating_inverse_cumulative_distribution_functions. GitHub repository.Google Scholar
[66] Sheridan-Methven Oliver. 2020. Getting started with approximate random variables: a brief guide for practitioners. https://github.com/oliversheridanmethven/approximate_random_variables. GitHub repository.Google Scholar
[67] Sheridan-Methven Oliver. 2021. Nested multilevel Monte Carlo methods and a modified Euler-Maruyama scheme utilising approximate Gaussian random variables suitable for vectorised hardware and low-precisions. DPhil thesis, Mathematical Institute, University of Oxford.Google Scholar
[68] Stallman Richard M. and community the GCC developer. 2020. Using the GNU compiler collection. https://gcc.gnu.org/onlinedocs/gcc-10.1.0/gcc/. Version 10.1.0.Google Scholar
[69] Stephens Nigel, Biles Stuart, Boettcher Matthias, Eapen Jacob, Eyole Mbou, Gabrielli Giacomo, Horsnell Matt, Magklis Grigorios, Martinez Alejandro, Premillieu Nathanael, Reid Alastair, Rico Alejandro, and Walker Paul. 2017. The ARM scalable vector extension. IEEE Micro 37, 2 (2017), 26–39.Google ScholarDigital Library
[70] Tezuka Shu. 1995. Uniform Random Numbers: Theory and Practice. Kluwer.Google ScholarCross Ref
[71] Pas Ruud van der, Stotzer Eric, and Terboven Christian. 2017. Using OpenMP — the Next Step: Affinity, Accelerators, Tasking, and SIMD. MIT Press.Google Scholar
[72] Virtanen Pauli, Gommers Ralf, Oliphant Travis E., Haberland Matt, Reddy Tyler, Cournapeau David, Burovski Evgeni, Peterson Pearu, Weckesser Warren, Bright Jonathan, Walt Stéfan J. van der, Brett Matthew, Wilson Joshua, Millman K. Jarrod, Mayorov Nikolay, Nelson Andrew R. J., Jones Eric, Kern Robert, Larson Eric, Carey C. J., Polat İlhan, Feng Yu, Moore Eric W., VanderPlas Jake, Laxalde Denis, Perktold Josef, Cimrman Robert, Henriksen Ian, Quintero E. A., Harris Charles R., Archibald Anne M., Ribeiro Antônio H., Pedregosa Fabian, Mulbregt Paul van, the SciPy 1.0 contributors, et al. 2020. SciPy 1.0: Fundamental algorithms for scientific computing in Python. Nature Methods 17 (2020), 261–272.Google ScholarCross Ref
[73] Wichura Michael J.. 1988. Algorithm AS 241: The percentage points of the normal distribution. Journal of the Royal Statistical Society. Series C (Applied Statistics) 37, 3 (April 1988), 477–484.Google ScholarCross Ref
[74] Wilson Edwin B. and Hilferty Margaret M.. 1931. The distribution of chi-square. Proceedings of the National Academy of Sciences of the United States of America 17, 12 (1931), 684.Google ScholarCross Ref
[75] Wittmann Markus, Zeiser Thomas, Hager Georg, and Wellein Gerhard. 2015. Short note on costs of floating point operations on current x86-64 architectures: Denormals, overflow, underflow, and division by zero. arXiv preprint arXiv:1506.03997 (2015).Google Scholar
[76] Xu Linlin and Ökten Giray. 2015. High-performance financial simulation using randomized quasi-Monte Carlo methods. Quantitative Finance 15, 8 (2015), 1425–1436.Google ScholarCross Ref

Index Terms

Approximating Inverse Cumulative Distribution Functions to Produce Approximate Random Variables
1. Computing methodologies
  1. Modeling and simulation
    1. Simulation types and techniques
      1. Massively parallel and high-performance simulations
      2. Uncertainty quantification
  2. Parallel computing methodologies
    1. Parallel algorithms
      1. Vector / streaming algorithms
2. Mathematics of computing

Recommendations

An approximate method for generating symmetric random variables

A method for generating values of continuous symmetric random variables that is relatively fast, requires essentially no computer memory, and is easy to use is developed. The method, which uses a uniform zero-one random number source, is based on the ...
Read More
An approximate method for generating asymmetric random variables

Tukey's lambda distribution is generalized to provide an algorithm for generating values of unimodal asymmetric random variables. This algorithm has the same advantages as the symmetric random variable generator previously given by the authors, except ...
Read More
Simple General Approximations for a Random Variable and its Inverse Distribution Function Based on Linear Transformations of a Nonskewed Variate

Linear transformations of a nonskewed random variable are employed to derive simple general approximations for a random variable having known cumulants. Introducing the unit normal variate, these become linear normal approximations.Some nonskewed ...
Read More

Comments

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

Published in
ACM Transactions on Mathematical Software Volume 49, Issue 3
September 2023
200 pages
ISSN:0098-3500
EISSN:1557-7295
DOI:10.1145/3624972
Editors:
Zhaojun Bai
University of California at Davis, USA
,
Wolfgang Bangerth
Colorado State University, USA
Issue’s Table of Contents
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].
Sponsors
In-Cooperation
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
- Published: 19 September 2023
- Online AM: 17 June 2023
- Accepted: 7 June 2023
- Revised: 25 May 2023
- Received: 22 December 2022
Published in toms Volume 49, Issue 3

Permissions
Request permissions about this article.
Request Permissions

Check for updates
Author Tags
Approximations
random variables
inverse cumulative distribution functions
random number generation
the Gaussian distribution
geometric Brownian motion
the Cox-Ingersoll-Ross process
the non-central χ2 distribution
multilevel Monte Carlo
the Euler-Maruyama scheme
the Milstein scheme
and high-performance computing
Qualifiers
- research-article
Conference
Funding Sources
Other Metrics
View Article Metrics

Article Metrics
- 0
  Total Citations
  View Citations
- 270
  Total Downloads
- Downloads (Last 12 months)270
- Downloads (Last 6 weeks)24
Other Metrics
View Author Metrics
Cited By
This publication has not been cited yet

PDF Format

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Full Text

View this article in Full Text.

View Full Text

Approximating Inverse Cumulative Distribution Functions to Produce Approximate Random Variables

ACM Transactions on Mathematical Software

Abstract

REFERENCES

Cited By

Index Terms

Recommendations

An approximate method for generating symmetric random variables

An approximate method for generating asymmetric random variables

Simple General Approximations for a Random Variable and its Inverse Distribution Function Based on Linear Transformations of a Nonskewed Variate