[Topic-models] Non-parametric topic models

Thibaut Thonet thibaut.thonet at irit.fr
Wed Feb 22 12:20:01 EST 2017


Hi Wray,

Thanks for your comments. I was thinking someone might have done that 
comparison (and publicize its results) just out of curiosity or for the 
sake of pedagogy. However I admit that pedagogy isn't always most 
productive/valued in research. I believe topic models would nonetheless 
need more pedagogical resources because I've been recently seeing more 
and more papers accepted or under review in top venues that contain 
serious and recurrent technical issues or unsaid (unaware?) 
approximations -- mostly in Gibbs samplers. But that is another topic ;-).

Best,

Thibaut

Le 21/02/2017 à 23:04, Wray Buntine a écrit :
>
> On 22 February 2017 at 02:57, Thibaut Thonet <thibaut.thonet at irit.fr 
> <mailto:thibaut.thonet at irit.fr>> wrote:
>
>     Hi Wray,
>
>     Thanks a lot for your detailed and thorough answer. So I conclude
>     from what you said that the model I described isn't 'wrong', but
>     it would just (most likely) perform worse than, e.g., HDP-LDA or
>     the NP-LDA from your 2014 KDD paper. I'm nonetheless surprised
>     that no work in literature evaluated this model and compared it
>     against hierarchical non-parametric models and against vanilla LDA
>     (symmetric-symmetric).
>
> Well, the words "meta", "infinite" and "hierarchical" earn a lot of 
> brownie points during the paper review process ;-)
> Actually, I think reviewers would treat it is too small an improvement 
> to make the big conferences.  The original
> HDP-LDA paper was a real revolution in capability.
>
>     Although it is indeed pretty sure that it would yield a higher
>     perplexity than that of hierarchical non-parametric models, it
>     seems that posterior inference for that model (e.g., using direct
>     assignment sampling), would be time-wise about as efficient as
>     that of vanilla LDA -- since table counts need not be sampled in
>     that version, given its non-hierarchical nature. So I'm curious
>     whether its effectiveness (perplexity, topic coherence) is better
>     than that of vanilla LDA, or otherwise if flat priors are more
>     penalizing in a non-parametric setting.
>
> I think you are right, it should perform well.  It should be fast too, 
> as the marginal posterior is similar to the standard one
> for the Dirichlet case.   But you would want to sample the 
> concentration parameter too.
>
>     Best,
>
>     Thibaut
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.cs.princeton.edu/pipermail/topic-models/attachments/20170222/03a18041/attachment.html>


More information about the Topic-models mailing list