Chung-chieh Shan
ABSTRACT
The ways we reason about probability distributions and explore their applications have been naturally shifting: away from thoughtful proving with definitions using first principles, and towards mechanical calculation with expressions using derived principles. This talk reviews three useful operations on distributions that we have started to express using equational derivations and even to automate as program transformations. These operations are (1) to recognize a density function as belonging to a known distribution family, (2) to eliminate an unused random variable by summation or integration, and (3) to disintegrate a joint measure into a marginal and a conditional measure. It is thus promising to support probabilistic reasoning by drawing techniques from both programming languages and computer algebra. Ongoing challenges include how to handle a wide variety of container data types and generating programs, and how human guidance should interact with machine assistance.
- Hadi Mohasel Afshar, Scott Sanner, and Christfried Webers. 2016. Closed-Form Gibbs Sampling for Graphical Models with Algebraic Constraints. In Proceedings of the 30th AAAI Conference on Artificial Intelligence. Google Scholar
Digital Library
- Bruno Buchberger. 2006. Mathematical Theory Exploration. In Automated Reasoning: 3rd International Joint Conference (Lecture Notes in Computer Science), Ulrich Furbach and Natarajan Shankar (Eds.). Springer, 1--2. Invited talk. Google ScholarDigital Library
- Wray L. Buntine. 1994. Operations for Learning with Graphical Models. Journal of Artificial Intelligence Research 2 (1994), 159--225. Google ScholarDigital Library
- Jacques Carette and Chung-chieh Shan. 2016. Simplifying Probabilistic Programs Using Computer Algebra. In Practical Aspects of Declarative Languages: 18th International Symposium, PADL 2016 (Lecture Notes in Computer Science), Marco Gavanelli and John H. Reppy (Eds.). Springer, 135--152.Google Scholar
- Sean R. Eddy. 2004. What is Bayesian Statistics? Nature Biotechnology 22, 9 (Sept. 2004), 1177--1178.Google ScholarCross Ref
- Thomas S. Ferguson. 1973. A Bayesian Analysis of Some Nonparametric Problems. The Annals of Statistics 1, 2 (March 1973), 209--230.Google ScholarCross Ref
- Michèle Giry. 1982. A Categorical Approach to Probability Theory. In Categorical Aspects of Topology and Analysis: Proceedings of an International Conference Held at Carleton University, Ottawa, August 11--15, 1981 (Lecture Notes in Mathematics), Bernhard Banaschewski (Ed.). Springer, 68--85.Google ScholarCross Ref
- Noah D. Goodman, Vikash K. Mansinghka, Daniel Roy, Keith Bonawitz, and Joshua B. Tenenbaum. 2008. Church: A Language for Generative Models. In Proceedings of the 24th Conference on Uncertainty in Artificial Intelligence, David Allen McAllester and Petri Myllymäki (Eds.). AUAI Press, 220--229. Google ScholarDigital Library
- Manuel Kauers. 2013. The Holonomic Toolkit. In Computer Algebra in Quantum Field Theory: Integration, Summation and Special Functions, Carsten Schneider and Johannes Blümlein (Eds.). Springer, 119--144.Google Scholar
- Peter S. Maybeck. 1979. Stochastic Models, Estimation, and Control. Number 141 in Mathematics in Science and Engineering. Academic Press.Google Scholar
- Jeffrey W. Miller and Matthew T. Harrison. 2018. Mixture Models with a Prior on the Number of Components. J. Amer. Statist. Assoc. 113, 521 (2018), 340--356.Google ScholarCross Ref
- Praveen Narayanan, Jacques Carette, Wren Romano, Chung-chieh Shan, and Robert Zinkov. 2016. Probabilistic Inference by Program Transformation in Hakaru (System Description). In Proceedings of FLOPS 2016: 13th International Symposium on Functional and Logic Programming (Lecture Notes in Computer Science), Oleg Kiselyov and Andy King (Eds.). Springer, 62--79.Google ScholarCross Ref
- Praveen Narayanan and Chung-chieh Shan. 2017. Symbolic Conditioning of Arrays in Probabilistic Programs. Proceedings of the ACM on Programming Languages 1, ICFP (2017), 11:1--11:25. Google ScholarDigital Library
- David Lorge Parnas. 1976. On the Design and Development of Program Families. IEEE Transactions on Software Engineering SE-2, 1 (March 1976), 1--9. Google ScholarDigital Library
- Judea Pearl. 1988. Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference. Morgan Kaufmann. Google ScholarDigital Library
- Norman Ramsey and Avi Pfeffer. 2002. Stochastic Lambda Calculus and Monads of Probability Distributions. In POPL '02: Conference Record of the Annual ACM Symposium on Principles of Programming Languages. ACM Press, 154--165. Google ScholarDigital Library
- Chung-chieh Shan and Norman Ramsey. 2017. Exact Bayesian Inference by Symbolic Disintegration. In POPL '17: Conference Record of the Annual ACM Symposium on Principles of Programming Languages. ACM Press, 130--144. Google ScholarDigital Library
- Sam Staton. 2017. Commutative Semantics for Probabilistic Programming. In Programming Languages and Systems: Proceedings of ESOP 2017, 26th European Symposium on Programming (Lecture Notes in Computer Science), Yang Hongseok (Ed.). Springer, 855--879. Google ScholarDigital Library
- Robert Zinkov and Chung-chieh Shan. 2017. Composing Inference Algorithms as Program Transformations. In Proceedings of the 33rd Conference on Uncertainty in Artificial Intelligence, Gal Elidan, Kristian Kersting, and Alexander T. Ihler (Eds.). AUAI Press.Google Scholar
Index Terms
- Calculating Distributions
Recommendations
Exact Bayesian inference by symbolic disintegration
POPL '17: Proceedings of the 44th ACM SIGPLAN Symposium on Principles of Programming LanguagesBayesian inference, of posterior knowledge from prior knowledge and observed evidence, is typically defined by Bayes's rule, which says the posterior multiplied by the probability of an observation equals a joint probability. But the observation of a ...
Exact Bayesian inference by symbolic disintegration
POPL '17Bayesian inference, of posterior knowledge from prior knowledge and observed evidence, is typically defined by Bayes's rule, which says the posterior multiplied by the probability of an observation equals a joint probability. But the observation of a ...
Expressions for Rényi and Shannon entropies for bivariate distributions
Exact forms of Rényi and Shannon entropies are determined for 27 continuous bivariate distributions, including the Kotz type distribution, truncated normal distribution, distributions with normal and centered normal conditionals, natural exponential ...
Comments