Sandro Stucki
Nada Amin
ABSTRACT
Probabilistic programming is a powerful high-level paradigm for probabilistic modeling and inference. We present Odds, a small domain-specific language (DSL) for probabilistic programming, embedded in Scala. Odds provides first-class support for random variables and probabilistic choice, while reusing Scala's abstraction and modularity facilities for composing probabilistic computations and for executing deterministic program parts. Odds accurately represents possibly dependent random variables using a probability monad that models committed choice. This monadic representation of probabilistic models can be combined with a range of inference procedures. We present engines for exact inference, rejection sampling and importance sampling with look-ahead, but other types of solvers are conceivable as well. We evaluate Odds on several non-trivial probabilistic programs from the literature and we demonstrate how the basic probabilistic primitives can be used to build higher-level abstractions, such as rule-based logic programming facilities, using advanced Scala features.
- PyMC: Bayesian inference in Python. Online, 2012.Google Scholar
- K. J. Brown, A. K. Sujeeth, H. Lee, T. Rompf, H. Chafi, M. Odersky, and K. Olukotun. A heterogeneous parallel framework for domain-specific languages. PACT, 2011. Google ScholarDigital Library
- M. Erwig and S. Kollmansberger. Functional pearls: Probabilistic functional programming in haskell. J. Funct. Program., 16(1):21--34, 2006. Google ScholarDigital Library
- S. Fischer, O. Kiselyov, and C.-c. Shan. Purely functional lazy non-deterministic programming. In Proceedings of the 14th ACM SIGPLAN international conference on Functional programming, ICFP '09, pages 11--22, New York, NY, USA, 2009. ACM. Google ScholarDigital Library
- J. Gibbons. Unifying theories of programming with monads. In Unifying Theories of Programming, volume 7681, pages 23--67. Springer, 2013.Google ScholarCross Ref
- N. D. Goodman, V. K. Mansinghka, D. M. Roy, K. Bonawitz, and J. B. Tenenbaum. Church: a language for generative models. In Proc. of Uncertainty in Artificial Intelligence, 2008.Google Scholar
- O. Kiselyov and C.-c. Shan. Embedded probabilistic programming. In W. M. Taha, editor, Domain-Specific Languages, volume 5658 of Lecture Notes in Computer Science, pages 360--384. Springer, 2009. Google ScholarDigital Library
- H. Lee, K. J. Brown, A. K. Sujeeth, H. Chafi, T. Rompf, M. Odersky, and K. Olukotun. Implementing domain-specific languages for heterogeneous parallel computing. IEEE Micro, 31(5):42--53, 2011. Google ScholarDigital Library
- D. J. Lunn, A. Thomas, N. Best, and D. Spiegelhalter. WinBUGS - A Bayesian modelling framework: Concepts, structure, and extensibility. Statistics and Computing, 10(4):325--337, Oct. 2000. Google ScholarDigital Library
- A. McCallum, K. Schultz, and S. Singh. Factorie: Probabilistic programming via imperatively defined factor graphs. In Adv. in Neural Inform. Processing Syst., volume 22, 2009.Google Scholar
- B. Milch, B. Marthi, S. Russell, D. Sontag, D. L. Ong, and A. Kolobov. BLOG: Probabilistic models with unknown objects. In L. Getoor and B. Taskar, editors, Statistical Relational Learning. MIT Press, 2007.Google Scholar
- B. Milch and S. Russell. General-purpose MCMC inference over relational structures. In Proc. 22nd Conference on Uncertainty in Artificial Intelligence, pages 349--358, 2006.Google Scholar
- T. Minka, J. Winn, J. Guiver, and D. Knowles. Infer.NET 2.5, 2012. Microsoft Research Cambridge. http://research.microsoft.com/infernet.Google Scholar
- A. Pfeffer. A general importance sampling algorithm for probabilistic programs. Tech. Rep. TR-12-07. Technical report, Harvard University, 2009.Google Scholar
- A. Pfeffer. Creating and manipulating probabilistic programs in figaro. UAI Workshop on Statistical Relational AI (StarAI), 2012.Google Scholar
- N. Ramsey and A. Pfeffer. Stochastic lambda calculus and monads of probability distributions. Proc. of the 29th ACM SIGPLAN-SIGACT Symp. on Principles of Program. Lang., pages 154--165, 2002. Google ScholarDigital Library
- T. Rompf, N. Amin, A. Moors, P. Haller, and M. Odersky. Scala-virtualized: Linguistic reuse for deep embeddings. In Higher-Order and Symbolic Computation (Special issue for PEPM'12, to appear).Google Scholar
- T. Rompf and M. Odersky. Lightweight modular staging: a pragmatic approach to runtime code generation and compiled dsls. Commun. ACM, 55(6):121--130, 2012. Google ScholarDigital Library
- T. Rompf, A. K. Sujeeth, H. Lee, K. J. Brown, H. Chafi, M. Odersky, and K. Olukotun. Building-blocks for performance oriented DSLs. DSL, 2011.Google ScholarCross Ref
- T. Sato. Generative modeling by prism. In ICLP, pages 24--35, 2009. Google ScholarDigital Library
- A. K. Sujeeth, H. Lee, K. J. Brown, T. Rompf, M. Wu, A. R. Atreya, M. Odersky, and K. Olukotun. OptiML: an implicitly parallel domain-specific language for machine learning. In Proceedings of the 28th International Conference on Machine Learning, ICML, 2011.Google Scholar
Index Terms
- What are the Odds?: probabilistic programming in Scala
Recommendations
Everything old is new again: quoted domain-specific languages
PEPM '16: Proceedings of the 2016 ACM SIGPLAN Workshop on Partial Evaluation and Program ManipulationWe describe a new approach to implementing Domain-Specific Languages(DSLs), called Quoted DSLs (QDSLs), that is inspired by two old ideas:quasi-quotation, from McCarthy's Lisp of 1960, and the subformula principle of normal proofs, from Gentzen's ...
Rewriting a shallow DSL using a GHC compiler extension
GPCE 2017: Proceedings of the 16th ACM SIGPLAN International Conference on Generative Programming: Concepts and ExperiencesEmbedded Domain Specific Languages are a powerful tool for developing customized languages to fit specific problem domains. Shallow EDSLs allow a programmer to program using many of the features of a host language and its syntax, but sacrifice ...
Rewriting a shallow DSL using a GHC compiler extension
GPCE '17Embedded Domain Specific Languages are a powerful tool for developing customized languages to fit specific problem domains. Shallow EDSLs allow a programmer to program using many of the features of a host language and its syntax, but sacrifice ...
Comments