[Topic-models] Does word-ordering matter in Gibbs sampling?

Eric Kang erickangnz at gmail.com
Wed Jul 26 00:49:38 EDT 2017


Hi everyone,

My apologies if this is an uninformed question, but in Gibbs sampling for LDA inference, aren’t the various counts of word-topic assignments updated word-by-word? Doesn’t this make it somewhat dependent on word ordering? For example, if word_1 is strongly associated with topic_1 and word_2 is strongly associated with topic_2, if I see a document {word_1, word_1, … (100 times), word_2, word_2, … (100 times), word_2}, then by the time I start seeing word_2, wouldn’t the algorithm be more inclined to think that it should be assigned to topic_1, compared to a scenario where I see the document {word_1, word_2, word_1, word_2, …}?

Thank you,
Eric


More information about the Topic-models mailing list