[Topic-models] Topic coherence in author-topic models

Jordan Boyd-Graber jordanbg at gmail.com
Tue Dec 6 08:47:26 EST 2016


Exactly; it would also be good to verify that those also correlate with
human judgements of author-word relationships.

--------------------
Jordan Boyd-Graber

Voice: 920.524.9464
jordanbg at gmail.com
http://boydgraber.org
--------------------

On Tue, Dec 6, 2016 at 3:16 AM, Ólavur Mortensen <olavurmortensen at gmail.com>
wrote:

> Thank you for the reply Jordan.
>
> PMI based on co-author occurrence would be interesting to try, but aren't
> these measures usually validated by using the correlation between the
> computed topic coherence and human annotators judgement of topic coherence?
>
> On Mon, Dec 5, 2016 at 5:25 PM Jordan Boyd-Graber <jordanbg at gmail.com>
> wrote:
>
>>
>> I'd guess it's perfectly valid to compute a topic coherence score in the
>> usual way, but it would tell you nothing about whether it matches up to the
>> *author*.  I suppose you could try doing something like PMI between author
>> and the words collected on a separate held out corpus.
>>
>> --------------------
>> Jordan Boyd-Graber
>>
>> Voice: 920.524.9464 <(920)%20524-9464>
>> jordanbg at gmail.com
>> http://boydgraber.org
>> --------------------
>>
>> On Sun, Dec 4, 2016 at 4:30 AM, Ólavur Mortensen <
>> olavurmortensen at gmail.com> wrote:
>>
>> I’m wondering how to compute topic coherence in the author-topic model
>> <https://mimno.infosci.cornell.edu/info6150/readings/398.pdf>. Any
>> information or advice would be appreciated.
>>
>> In case it’s needed, a very short description of author-topic models and
>> topic coherence:
>>
>>    - An author-topic model is basically just a topic model where each
>>    author is associated with a distribution over topics. It is based on Latent
>>    Dirichlet Allocation.
>>    - Topic coherence <http://dirichlet.net/pdf/mimno11optimizing.pdf>
>>    evaluates the quality of topics by a metric based on the co-document
>>    frequency of words.
>>
>> Is topic coherence valid for the author-topic model?
>>
>> Is there any different metric I could use to evaluate the topics
>> resulting from a author-topic model? For example, one could use the
>> co-author frequency of words. Is there any annotated data (like in the
>> topic coherence paper) that can be use to validate a topic coherence metric
>> in the author-topic model?
>>>>
>> _______________________________________________
>> Topic-models mailing list
>> Topic-models at lists.cs.princeton.edu
>> https://lists.cs.princeton.edu/mailman/listinfo/topic-models
>>
>>
>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.cs.princeton.edu/pipermail/topic-models/attachments/20161206/2c991f0c/attachment.html>


More information about the Topic-models mailing list