Chong Wang will present his research seminar/general exam on Wednesday Jan 21 at 2PM in Room 402. The members of his committee are: David Blei (advisor), Fei-Fei Li, and Rob Schapire. Everyone is invited to attend his talk, and those faculty wishing to remain for the oral exam following are welcome to do so. His abstract and reading list follow below. -------------------------------------------- Learning topic models from multiple corpora Abstract In this talk, we consider the problem of learning topic models from multiple corpora. Examples of multi-corpora data include news articles from different times or different locations, and scientific papers from different conferences or different years. The majority of topic models, however, usually ignore this addition information. Simply combining multiple corpora into one large corpus or treating each corpus individually can't provide the ability to analyze the high-level relations among multiple corpora, e.g., are topics similar or different from time to time or from location to location? How are they related to each other? I will describe two new approaches for learning topic models from multiple corpora: continuous time dynamic topic model (cDTM) and Markov topic model(MTM). In cDTMs, documents from the same time point are considered as a corpus. The cDTM extends dynamic topic models (DTMs) by using Brownian motion to model the latent topics through the time line. We derive an efficient variational approximate inference algorithm that takes advantage of the sparsity of observations in text, a property that lets us easily handle many time points. Thus, cDTM is able to discover topic evolutions in a much finer time resolution. In MTMs, papers from the same conference are treated as a corpus. Then we apply Gaussian (Markov) random fields to model the correlations of different corpora. MTMs capture both the internal topic structure within each corpus and the relationships between topics across the corpora. In addition, we will show cDTMs and DTMs can be formulated as special cases of MTMs. Quantitative results and qualitative discoveries (interesting topic patterns) will also be presented. Books: 1) Pattern Recognition and Machine Learning, by Christopher M. Bishop, Springer, 2006 chapters: 1, 2, 3.1-3.5, 4, 5, 8, 9, 10, 11.1-11.3, 12.1-12.2, 13, 14. 2) An Introduction to Probabilistic Graphical Models (unpublished manuscript), by Michael I. Jordan, 2002 chapters: 2, 3, 4, 5, 8, 9, 10, 11, 12, 13, 14, 15 4) (Optional) Artificial Intelligence: A Modern Approach, by Stuart Russell and Peter Norvig, Prentice Hall Series in Artificial Intelligence, 2003 chapters: 3.1-3.5, 4.1-4.3, 13, 14, 15.1-15.5, 18.1-18.3, 20.1-20.5, 23.2-23.3 Papers: 1) D. Blei, A. Ng, and M. Jordan. Latent Dirichlet allocation. JMLR, 3:993-1022, 2003. 2) D. Blei and J. Lafferty. Dynamic topic models. In ICML, 2006. 3) T. Griffiths and M. Steyvers. Finding scientific topics. Proc Natl Acad Sci USA, 101 Suppl 1:5228-5235, April 2004. 4) C. Andrieu, N. de Freitas, A. Doucet, and M. Jordan. An introduction to MCMC for machine learning. Machine Learning, 50(1):5-43, January 2003. 5) M. Jordan, Z. Ghahramani, T. Jaakkola, and L. Saul. An introduction to variational methods for graphical models. Machine Learning, 37(2):183-233, 1999. 6) Y. Teh, M. Jordan, M. Beal, and D. Blei. Hierarchical Dirichlet processes. Journal of the American Statistical Association, 2006. 7) L. Fei-Fei and P. Perona. A Bayesian hierarchical model for learning natural scene categories. In CVPR, 2005. 8) Havard Rue and Turid Follestad, Gaussian Markov Random field models with applications in spatial statistics, preprint, 2003 9) D. Blei and J. Lafferty. Correlated topic models. In NIPS, 2005. 10) R. Kalman. A new approach to linear filtering and prediction problems. Transaction of the AMSE: Journal of Basic Engineering, 82:35-45, 1960. 11) L. Ruschendorf. Convergence of the iterative proportional fitting procedure. The Annals of Statistics, 23(4):1160-1174, 1995. 12) X. Wang and A. McCallum. Topics over time: a non-Markov continuous-time model of topical trends. In KDD, 2006.