Functional Probabilistic Programming

Pfeffer, A.

Presented at the Commercial Users of Functional Programming workshop, affiliated with the International Conference on Functional Programming, Boston, MA. (September 2013)

Probabilistic modeling is one of the most widely used approaches to machine learning (ML). In recent years, the number and variety of probabilistic models has increased dramatically. Currently, developing a new probabilistic model requires developing a new representation and reasoning and learning algorithms. This is expensive and requires much expertise. Probabilistic programming (PP) is designed to make development of probabilistic ML applications less expensive and easier for non-experts. A PP language provides the tools to represent complex models using the power of programming languages and provide reasoning and learning algorithms that work automatically for any model written in the language. The development of PP languages has bee inspired by functional programming. Just as an expression in a function programming language defines a computation that produces a value, an expression in a functional PP language defines an expression that stochastically produces a value by sampling the program. The purpose of PP, however, is not just to produce stochastic values, but to reason about the probability distribution over values produced by the program, for example, to predict the value of one variable given another.

In this talk, I will describe PP through two functional PP languages I have developed. The first language, IBAL, was the first general-purpose PP language. Implemented in OCaml, it showed how probabilistic reasoning could be performed with algebraic data types and higher-order functions. Our more recent language, Figaro, is an embedded library in Scala. The main concept behind Figaro is that a Scala program is used to construct a Figaro data structure that represents a functional probabilistic program. This enables creation, manipulation, and integration of probabilistic programs in a way that was not previously feasible. After describing the basic Figaro concepts, I will present several examples highlighting the power of Figaro.

For More Information

To learn more or request a copy of a paper (if available), contact Avi Pfeffer.

(Please include your name, address, organization, and the paper reference. Requests without this information will not be honored.)