LDA and Twitter-LDA

LDA
● Algorithm
– Initialize topic assignments randomly
– For each iteration
● For each document
– For each word
● Re-sample a topic for that word, given all the other words

and their current topic assignments

 LDA T1 kw11,kw12,...
T2 kw21,kw22,..
T3 kw31,kw32,..
● Plate notation
.. ...

For each of the K topic,

a distribution over words

– Each circle is a variable, each box is a “for each” loop

– D plate denotes the collection of documents
– N plate denotes the collection of words within a doc
 Assign topcs

3 2 1 3 1
commission trade nbn abc union

3 2 1 3 1
commission trade nbn abc union
 3 2 1 3 1
commission trade nbn abc union

Topic 1 Topic 2 Topic 3

commission 1 2 20
union 1 44 10
nbn 10 20 1
abc 2 5 50
trade 10 8 1

….

Total counts from all docs

 3 2 1 3 1
commission trade nbn abc union

Topic 1 Topic 2 Topic 3

commission 1 2 20
union 1 44 10
nbn 10 20 1
abc 2 5 50
trade 10 8 1

….
 re-sampling

3 ? 1 3 1
commission trade nbn abc union

Topic 1 Topic 2 Topic 3

commission 1 2 20
union 1 44 10
nbn 10 20 1
abc 2 5 50
trade 10 7 1

….
 Re-assigning a topic to “trade”
● Topic 1 Topic 2 Topic 3

3 ? 1 3 1
commission trade nbn abc union
 Re-assigning a topic to “trade”
● Topic 1 Topic 2 Topic 3

3 ? 1 3 1
commission trade nbn abc union

Topic 1 Topic 2 Topic 3

trade 10 7 1

Which topic likes the word “trade”

 Re-assigning a topic to “trade”
● Topic 1 Topic 2 Topic 3

Area of rect: how much doc likes the topic * how much the topic likes the word

3 ? 1 3 1
commission trade nbn abc union

Topic 1 Topic 2 Topic 3

trade 10 7 1
 Pick a topic for “trade”

3 ? 1 3 1
commission trade nbn abc union

Topic 1 Topic 2 Topic 3

commission 1 2 20
union 1 44 10
nbn 10 20 1
abc 2 5 50
trade 10 7 1

….
 Pick a topic for “trade”

3 1 1 3 1
commission trade nbn abc union

Topic 1 Topic 2 Topic 3

commission 1 2 20
union 1 44 10
nbn 10 20 1
abc 2 5 50
trade 11 7 1

….
 Twitter LDA
● algorithm