[Topic-models] Fwd: Some question about estimating hyperparameter of LDA

Feng Wang weafly at gmail.com
Fri May 30 05:34:52 EDT 2008


Hi, all

 Does anyone here has some tech reports or publications show their
results of eastimating hyper parameter of LDA.
Either using sampling methods or other methods, please share me the
URLs or documents.

Thanks,
Feng

2008/5/4 Laura Dietz <dietz at informatik.hu-berlin.de>:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> Hi Feng,
>
> we discussed this issue in Feb08 see the list archive:
> https://lists.cs.princeton.edu/pipermail/topic-models/2008-February/thread.html
>
> There are multiple approaches:
> - - use gradient descent for hyperparam estimation (i.e. see Minka) and
> combine it as a stochastic EM algorithm
> - - use a naive Metropolis Hastings steps (i.e. a random walk MH sampler)
> and interleave it with the gibbs steps.
> - - use adaptive rejection sampling
> - - YeeWhye Teh mentioned, that the auxiliary variable approach for
> sampling hyperparams of the Dirichlet process may also be applicable to
> the fixed-K setting.
>
> You may also sample asymetric hyperparams with most of the approaches.
>
> Best,
>        Laura
>
> wrote:
> | Hi, Gregor and All others,
> |
> |   I'm now trying to follow the work on estimating hyper-parameter of
> | LDA. As a beginner of LDA, I have read the turtorial "Parameter
> | estimation for text analysis"
> | written by Gregor Heinrich, and got some questions on it, following
> | are these questions:
> |
> | 1. Have anyone ever tried to using sampling method to estimate the
> | hyper-parameters? Why not estimate asymmetric prior instead symmetric
> | one, since asymmetric
> | sounds more reasonable.
> |
> | 2. Gregor has mentioned two sampling methods, adaptive rejection
> | Metropolis sampling and adaptive rejection method, as we know that the
> | latter one can skip the
> | computationally expensive Metropolis step but with the limitation of
> | requirement of (log f)'' <0 (log concave density), however, I cannot
> | prove that the p(/alpha| z) in
> | LDA is a log concave density. Does anyone ever give a prove of that?
> | If it cannot be proved, is it means that this sampling work cannot be
> | achieved by adaptive
> | rejection sampling?
> |
> | 3.In the section 6.3, Efficient estimation, Gregor talks about using
> | results of Mink00, and said that the result cannot be used to estimate
> | /beta, because of it on the condition
> | of {M, E{N}}>20K, but I really cannot find where does this condition
> | come from Mink's paper, and also I cannot figure out why /alpha meets
> | this condition? Did I miss something?
> |
> | 4.There're quite a few method in Mink's paper, why not use other's
> | such as Newton-Raphson which don't have condition limitation
> | aforementioned?
> |
> | anyone got any idea about any of these questions, please help me to
> | figure it out, thank you very much!
> |
> | Best,
> | Feng
> | _______________________________________________
> | Topic-models mailing list
> | Topic-models at lists.cs.princeton.edu
> | https://lists.cs.princeton.edu/mailman/listinfo/topic-models
>
>
> - --
> - ------------------------------------------------------
> Laura Dietz, Dipl.-Inform.
> Max Planck PhD scholarship holder
> Research Group 2 (Machine Learning)
> Max Planck Institut für Informatik
> Stuhlsatzenhausweg 85, 66123 Saarbrücken, Germany
> Room 429
> Phone: +49 681 9325 529
> E-mail: dietz at mpi-inf.mpg.de
> http://www.mpi-inf.mpg.de/~dietz
> http://www.mpi-sb.mpg.de/departments/rg2/
> - ------------------------------------------------------
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.7 (MingW32)
> Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org
>
> iD8DBQFIHb0Zu7dZjvX3AX8RAsnRAJ4+hGEVMXvYwBSS8mkNBC4CYaS0zACg57Wt
> 4F75qGIvMLANCZ47UnuMqeo=
> =EGna
> -----END PGP SIGNATURE-----
>


More information about the Topic-models mailing list