[talks] T Wang generals

[Melissa M. Lawson]

Tian Wang will present his research seminar/general exam on 
Tuesday May 15 at 10:30AM in Room 402.  The members of his 
committee are:  David Blei (advisor), Rob Schapire, and Rebecca
Fiebrink.  Everyone is invited to attend his talk and those 
faculty wishing to remain for the oral exam following are welcome 
to do so.  His abstract and reading list follow below.
--------------------------

Title:

Supervised Hierarchical Dirichlet Process

Abstract:

Hierarchical Dirichlet Process have been proved effective as a Bayesian non-parametric model for text document topic modeling. HDP places a non-parametric prior on distribution of topics, thus allowing the number of topics to be learnt through data. It works as a Bayesian non-parametric extension to Latent Dirichlet Allocation. In my research, I introduce Supervised Hierarchical Dirichlet Process, a Bayesian non-parametric extension to Supervised Latent Dirichlet Allocation. In topic modeling setting, Supervised Hierarchical Dirichlet Process relates a continuous response variable, typically a rating to a document, to the topic assignments of all words in a document. For learning I use Markov Chain Monte Carlo method to sample the intractable posterior. More specifically, I derived and implemented a collapsed Gibbs sampler for Supervised Hierarchal Dirichlet Process. I also derived posterior sampling equations for Supervised Latent Dirichlet Allocation, which is not published before or in any software package. Like Supervised Latent Dirichlet Allocation, the main application of this model is to predict document ratings given the documents. I use movie reviews and movie review ratings as my data set. Supervised Hierarchical Dirichlet Process is compared with a conventional Hierarchical Dirichlet Process followed by a separate regression with the response variable in terms of prediction error on new documents. 

Reading List:

[1]  C. Bishop. Pattern Recognition and Machine Learning (Information Science and Statistics), chapter 8,11. Springer-Verlag New York, Inc., Secaucus, NJ, USA, 2006. 
[2]  D. Blei and P. Frazier. Distance dependent chinese restaurant processes. Journal of Machine Learning Research, 12:2461–2488, 2011. 
[3]  D. Blei and M. Jordan. Variational methods for the dirichlet process. Journal of Bayesian Analysis, 1[1]:121–144, 2006. 
[4]  D. Blei, A. Ng, and M. Jordan. Latent dirichlet allocation. Journal of Machine Learning Research, 2003. 
1 
[5]  D. Blei and J. McAuliffe. Supervised topic models. In J.C. Platt, D. Koller, Y. Singer, and S. Roweis, editors, Advances in Neural Information Processing Systems 20, pages 121–128. MIT Press, Cambridge, MA, 2008. 
[6]  S. Gershman and D. Blei. A tutorial on bayesian nonparametric models. June 2011. 
[7]  R. Neal. Markov chain sampling methods for dirichlet process mixture models. JOURNAL OF COMPUTATIONAL AND GRAPHICAL STATISTICS, 9(2):249–265, 2000. 
[8]  Y. Teh, M. Jordan, M. Beal, and D. Blei. Hierarchical dirichlet processes. Journal of the American Statistical Association, 101[476]:1566–1581, 2006. 
[9]  Y. W. Teh. Dirichlet processes. In Encyclopedia of Machine Learning. Springer, 2010. 
[10]  C. Wang, J. Paisley, and D. Blei. Online variational inference for the hierarchical dirichlet process. Artificial Intelligence and Statistics, 2011.