[Ml-stat-talks] Advanced Methods in Probabilistic Modeling

David Blei blei at CS.Princeton.EDU
Tue Aug 23 14:45:52 EDT 2011

hi ml stat talks

i'm writing to announce my course for next semester, advanced topics
in probabilistic modeling.  we are going to cover variational
inference, hierarchical modeling, bayesian nonparametric methods, and
model specification.

it will be a lecture course and can be thought of as part 2 of cos513
"foundations of probabilistic models."  i put a description and
preliminary syllabus here:


as well as below my signature in this email.

note that there is no class size limit, and auditors are welcome.
(auditors, please do register.)

please get in touch with me if you haven't taken cos513 but would like
to take the course.  i'm happy to talk about whether you have the
right background and to suggest readings.



COS597C: Advanced Methods in Probabilistic Modeling

Fall, 2011
M/W, 11:00AM - 12:20PM
Location TBD
David M. Blei


We will study some advanced methods in probabilistic modeling that are
central to modern machine learning and statistics. We will focus on
four subjects:

posterior inference with variational methods
hierarchical modeling for grouped data
model selection, specification, and checking
Bayesian nonparametric modeling

We will emphasize algorithms and applications as well as the
theoretical underpinnings of these subjects.

Prerequisites and requirements

This course is appropriate for students who have taken COS513
"Foundations of Probabilistic Modeling" or who are familiar with the
material from that course. Contact David Blei if you are unsure about
whether this is the right course for you to take.

The course will consist of lectures and "practical" lectures. During
practical lectures, we will implement and explore the properties of
algorithms as a class. (We will learn and use R.)

The requirements are

Brief reading response papers (less than one page)
A final project
Class attendance and participation

Preliminary Syllabus

Introduction and review (2 lectures)

Introduction, course overview, course requirements
Review (graphical models, posterior inference, computation)
Variational inference (6 lectures)

Mean field variational inference

Practical: Mean field for mixtures of Gaussians
Variational inference in compound exponential families
Practical: Variational inference for Bayesian logistic regression
Wainwright and Jordan 2008
Stochastic optimization and online variational inference

Hierarchical modeling (5 lectures)

Introduction to hierarchical modeling
Supervised modeling: Hierarchical generalized linear models
Practical: Gibbs sampling for hierarchical GLMs
Unsupervised modeling: Mixed membership models
Practical: The mixed membership stochastic blockmodel

Bayesian nonparametrics (8 lectures)

Chinese restaurant process mixtures
Practical: Gibbs sampling for CRP mixtures
Random measures, Dirichlet process mixtures, gamma representations
Stick-breaking representations and variational inference
Practical: Variational inference for DP mixtures
Hierarchical Dirichlet processes
Practical: Gibbs sampling for HDP topic models
Gamma representations, completely random measures
The Indian Buffet process, the beta process, matrix factorization
Model specification (3 lectures)

Posterior predictive checks

Practical: Posterior predictive checks for HDPs
Statistics, Science, Philosophy: George Box, Gelman and Shalizi

More information about the Ml-stat-talks mailing list