BAYES

RULE
Applied Sta+s+cs and Compu+ng Lab Indian School of Business

Applied Sta+s+cs and Compu+ng Lab

"The essence of the Bayesian approach is to provide a mathema5cal rule explaining how you should change your exis5ng beliefs in the light of new evidence. In other words, it allows scien5sts to combine new data with their exis5ng knowledge or exper5se. The canonical example is to imagine that a precocious newborn observes his rst sunset, and wonders whether the sun will rise again or not. He assigns equal prior probabili5es to both possible outcomes, and represents this by placing one white and one black marble into a bag. The following day, when the sun rises, the child places another white marble in the bag. The probability that a marble plucked randomly from the bag will be white (i.e., the child's degree of belief in future sunrises) has thus gone from a half to two-thirds. AHer sunrise the next day, the child adds another white marble, and the probability (and thus the degree of belief) goes from two-thirds to three- quarters. And so on. Gradually, the ini5al belief that the sun is just as likely as not to rise each morning is modied to become a near-certainty that the sun will always rise. The Economist [Ar+cle "In praise of Bayes , Sept. 30 2000]

Applied Sta+s+cs and Compu+ng Lab

Learning Goals
Why we need Bayes Rule Understand the rule Learn to apply the rule

Applied Sta+s+cs and Compu+ng Lab

An Example
An electronics retailer adverEses his products through two media: Social Networking sites and Television AdverEsements. Through a market research, he learns that of the people that have viewed his ad 70% viewed it on a Social Networking site and 30% on the TV. Apart from this informaEon, he knows that 20% of the people who view his ads on Social Networking sites buy his products and 30% of the people who view his ads on TV buy his products. Now he wants to know which of the two media is more eecEve in promoEng his products? Where should he adverEse more to increase his sales?
Applied Sta+s+cs and Compu+ng Lab
4

SoluEon
Let us rst separate this informaEon into small pieces Several Events:
S: The ad is viewed on a Social networking site T: The ad is viewed on TV Buy: A person aware of the ad buys the product P(S)=0.7 P(T)=0.3 P(Buy|S)=0.2 P(Buy|T)=0.3

InformaEon given:

Applied Sta+s+cs and Compu+ng Lab

SoluEon (Contd)
To decide as to which medium of adverEsing is more eecEve we must rst have an idea of how likely it is that a person who purchases the product has viewed the ad through that medium i.e. we must know the following probabiliEes: P(S|Buy) P(T|Buy)
Applied Sta+s+cs and Compu+ng Lab
6

SoluEon (Contd)
Consider P(S|Buy), from our tutorial on CondiEonal Probability we have learnt that: P(S Buy) P(S | Buy) = P( Buy) Similarly, P ( S Buy ) can be wri_en as P(S Buy) = P( Buy | S ) P(S ) P( Buy | S ) P(S ) P ( S | Buy ) = So ,
P( Buy)

| S The numerator is: P ( Buy ) P ( S ) = 0.7 0.2 = 0.14 . (1) We are now half way to the answer
7

Applied Sta+s+cs and Compu+ng Lab

SoluEon (Contd)
Now, we need the value of P(Buy) Since Social Networking sites and TV ads are the only media the retailer uses, the events S & T form a parEEon of the Sample Space Consider the following Venn-diagram:

It is clear from this diagram that we can write Buy in the following manner: Buy=(Buy S) U (Buy T) [ Note that, (Buy S) (Buy T) = ]

P(Buy) = P((Buy S) + P(Buy T) [Law of Total Probability] P(Buy) = P(Buy|S) * P(S) + P(Buy|T)*P(T) (2) (from deniEon of condiEonal probability )

Applied Sta+s+cs and Compu+ng Lab

SoluEon (Contd) We now have the denominator as well: P(Buy) = P(Buy|S) * P(S) + P(Buy|T)*P(T) = 0.2*0.7+0.3*0.3 =0.23 There fore, the required probability is: | S ) P(S ) P( Buy | S ) P(S ) 0.14 (S | Buy) = P(Buy P = = 0.61 P( Buy) P( Buy | S ) P(S ) + P( Buy | T ) P(T ) 0.23 P( Buy | S ) P(S ) P ( S | Buy ) = Incidentally ( Buy | S ) P ) + P ( Buy | T ) P ( T ) is called P (S the Bayes Rule Applied Sta+s+cs and Compu+ng Lab

SoluEon (Contd)
Let us now calculate P(T|Buy) in a similar manner: P( Buy | T ) P(T ) P( Buy | T ) P(T ) 0.09 P(T | Buy) = = = 0.39 P( Buy) P( Buy | S ) P( S ) + P( Buy | T ) P(T ) 0.23 We now have both P(S|Buy) & P(T|Buy) So 61% of the people who buy the products have viewed the ad on a social networking site and 39% on TV ads Clearly, the retailers adverEsements on Social Networking sites were more eecEve This gives the retailer a good idea of where to concentrate his adverEsement budget to increase his sales

Applied Sta+s+cs and Compu+ng Lab

10

Bayes Rule: A formal DeniEon

For two events A and B, the Bayes Rule is
P( A | B ) P( B ) P( B | A) = P( A | B ) P( B ) + P( A | B c ) P( B c )

Consider the parEEon {Bj} of the Sample Space, then the Bayes rule can be extended as follows: P( A | Bi ) P( Bi ) P( Bi | A) =

P( A | B ) P( B )
j j j

Applied Sta+s+cs and Compu+ng Lab

11

Thank you

Applied Sta+s+cs and Compu+ng Lab