[Topic-models] Topic coherence in author-topic models

Ólavur Mortensen olavurmortensen at gmail.com
Tue Dec 6 05:16:50 EST 2016

Thank you for the reply Jordan.

PMI based on co-author occurrence would be interesting to try, but aren't
these measures usually validated by using the correlation between the
computed topic coherence and human annotators judgement of topic coherence?

On Mon, Dec 5, 2016 at 5:25 PM Jordan Boyd-Graber <jordanbg at gmail.com>

> I'd guess it's perfectly valid to compute a topic coherence score in the
> usual way, but it would tell you nothing about whether it matches up to the
> *author*.  I suppose you could try doing something like PMI between author
> and the words collected on a separate held out corpus.
> --------------------
> Jordan Boyd-Graber
> Voice: 920.524.9464 <(920)%20524-9464>
> jordanbg at gmail.com
> http://boydgraber.org
> --------------------
> On Sun, Dec 4, 2016 at 4:30 AM, Ólavur Mortensen <
> olavurmortensen at gmail.com> wrote:
> I’m wondering how to compute topic coherence in the author-topic model
> <https://mimno.infosci.cornell.edu/info6150/readings/398.pdf>. Any
> information or advice would be appreciated.
> In case it’s needed, a very short description of author-topic models and
> topic coherence:
>    - An author-topic model is basically just a topic model where each
>    author is associated with a distribution over topics. It is based on Latent
>    Dirichlet Allocation.
>    - Topic coherence <http://dirichlet.net/pdf/mimno11optimizing.pdf>
>    evaluates the quality of topics by a metric based on the co-document
>    frequency of words.
> Is topic coherence valid for the author-topic model?
> Is there any different metric I could use to evaluate the topics resulting
> from a author-topic model? For example, one could use the co-author
> frequency of words. Is there any annotated data (like in the topic
> coherence paper) that can be use to validate a topic coherence metric in
> the author-topic model?
> _______________________________________________
> Topic-models mailing list
> Topic-models at lists.cs.princeton.edu
> https://lists.cs.princeton.edu/mailman/listinfo/topic-models
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.cs.princeton.edu/pipermail/topic-models/attachments/20161206/02864863/attachment.html>

More information about the Topic-models mailing list