WinBUGS

Software for Bayesian Inference

OUTLINE
Brief overview of WinBUGS
Basics of Bayesian Inference
Navigating in WinBUGS
Examples
What is WinBUGS?
WinBUGS is software used for analysis of Bayesian Statistical
Models
Uses Markov Chain Monte Carlo (MCMC) techniques to obtain
estimates of posterior distributions
WinBUGS is a windows based version of older software called
BUGS.
BUGS stands for Bayesian inference Using Gibbs Sampling
WinBUGS is FREE (thats right, free)
WinBUGS handles simple to complex statistical models
WinBUGS is object-oriented. That is, sections of your code can be
written in somewhat random order. You highlight the code you want
to run before running it.
A Help manual comes with the software (available via the help
window)
There is a graphical mode of writing WinBUGS code that we wont
cover today, called Doodles.

Review of Bayesian Inference
Recall: Letting represent a parameter of interest, we specify a prior distribution
for . In addition, we specify the Likelihood of using standard inference methods.

Note: The Prior distribution of represents our prior beliefs about . A distribution
with a small variance says you have a lot of prior information about . Larger
variance implies little or no prior information about .

Inference about comes from the Posterior distribution of given Y.

The posterior distribution is computed from the prior and likelihood. Symbolically
we state posterior(|Y) prior() x likelihood(). That is posterior is proportional
to prior times likelihood. The constant of proportionality is a constant required to
make posterior(|Y) a valid probability density (that is, integrate to 1).

Sometimes, the posterior distribution can be calculated simply.

Example: Let have a gamma(a,b) prior, where a and b are known. Let Y be the
observed data, and let Y1, Y2, , Yn be iid from a Poisson() distribution. Then the
posterior of is proportional to.

( )
b
a
a
i
Y
n
e
b a Y
e
i
u
u
u
u

[
1
1
!
Bayesian Inference - continued
Simplifying somewhat reduces this expression to


which is a kernel of a Gamma ( ) distribution.

Often, however, the posterior does not easily reduce into a
recognizable form. The mathematics of computing the appropriate
posterior was a significant challenge for Bayesians before
computing became prevalent.

WinBUGS uses information about the likelihood and prior to
sample from the posterior distribution. With enough samples a
very good approximation to the posterior distribution is assembled.
( )

|
|
.
|

\
|
+

+ 1
1
exp ) tan (
a Y
i
nb
b
ts cons u
u
1
,
+
+

nb
b
a Y
i
WinBUGS Menus
Like in SAS, you will submit your programs (and
monitor output) through Menus.
Important WinBUGS Menus
Attributes: font attributes (size, color, etc.)
Model: allows you to submit your code; also allows
you to run MCMC samples to estimate the posterior
Inference: declares which parameter(s) or functions
you want to have monitored through the MCMC
process. All parameters for which you require
posterior estimates should be entered here.

WinBUGS program syntax
All WinBUGS programs require 3 sections:
Stating the likelihood and prior(s) (called the Model statement)
Entering the observed data
Entering a set of initial values for the parameters. Doing this gives the
MCMC algorithm a set of starting values for the parameters.

Syntax:
All defined variables will be either stochastic (random) or deterministic
(directly equal to some value or variable).
Specifying a stochastic variable requires the syntax ~
Example: If the variable X is a Poisson random variable with mean m, then you
would write X~dpois(m) in your program to define X.
Specifying a deterministic variable requires the syntax <-
Example: If the variable CV, a ratio of mean/std dev is required, then it could be
defined as CV<-mu/sigma

Program Syntax - continued
In WinBUGS, the symbols used to the start and end of sections of code are { and }.

Do loops are written as

for (j in 1:100)
{commandline1
commandline2
.
.
.
last command line
}

Arrays are written as [] after a variable name.
For example, given n observations on a variable Y, Yi would be written as Y[i] in the code.

Entering Data or Initial Values starts with a List command. Then, in parentheses,
you enter values for all required variables.
Example: To enter 5 observations for Y[i]; and the value 9.2 for a constant k, the syntax
would be
list(y=c(3, 5, 1, 0, 2),k=9.2)
Example 1
Continuing with our model from earlier, let
Y~Poisson() distribution.
Consider a Gamma(a,b) prior for
If we observe the following data for Y:
Y: 3, 5, 1, 0, 2, 1, 4, 3, 1, 6
We will let a=1 and b=2 for purposes of this example.
In reality these values would be chosen after careful
examination.
We want to analyze the posterior distribution of .
Now lets see how this program would be written
in WinBUGS
Example 1 - continued
Model
#STATE THE LIKELIHOOD
{
for (i in 1:10)
{
y[i]~dpois(lambda)
}

#PRIOR DISTRIBUTION FOR LAMBDA
lambda~dgamma(a,b)
}

#ENTER THE DATA
list(y=c(3, 5, 1, 0, 2, 1, 4, 3, 1, 6),a=1,b=0.5)

#SET INITIAL VALUES FOR LAMBDA
list(lambda=2)
list(lambda=8)
list(lambda=0)

Example 1 - continued
To submit this program:

1. Highlight the part of the code starting with model, through at least the first { symbol.

2. Click the Model menu and select the Specification option.

3. Click Check Model. If the likelihood is written correctly, a message at the bottom of the screen appears
saying Model is syntactically correct.

4. Highlight the data part of the code, starting at the word list.

5. Click the Load Data button. You should see a confirmation message stating Data Loaded at the
bottom of the screen.

6. Enter 3 in the Number of Chains field. Click the Compile Button. You should receive the message
Model Compiled.

7. Highlight the first initial value. Here this says list(lambda=2). Click the Load Inits button. Continue
similarly for the 2
nd
and 3
rd
initial values, clicking Load Inits after highlighting each list one at a time.

8. Next click the Inference Menu, and then the Samples option. In the Node field type lambda. Then
click set.

9. Finally, click the Model Menu, and then the Update option. Here, you tell WinBUGS how many
MCMC updates (samples) to run to estimate the posterior. You should choose at least 1000 updates in
general. Since this code runs quickly, enter 5000 updates. Then click Update.

Example 1 - continued
Viewing Posterior Information
In the Sample Monitor Tool
window, click Lambda as the node.

To view the posterior density of
Lambda, click the Density button.

To get mean, std dev, median, and
percentiles for the posterior, click
Stats.

We should also check the History
option to ensure the 3 MCMC chains
have converged. In this window,
each chain has a separate color.
Ideally, you should see all the colors
mixing together well.

As a check, we expected our
posterior distribution to be
Gamma(27, 2/21).

lambda chains 1:3 sample: 15000
0.0 2.0 4.0
0.0
0.25
0.5
0.75
1.0
Output should look like:
node mean sd MC error 2.5% median 97.5% start sample
lambda 2.57 0.4976 0.004184 1.695 2.535 3.642 1 15000
lambda chains 1:3
iteration
1 2000 4000
1.0
2.0
3.0
4.0
5.0
Example 1 - continued
Expected Results:
Given what we know about Gamma(27,2/21)
distributions:
Mean: 2.571
Std Deviation: 0.2449
2.5
th
percentile: 1.695
97.5
th
percentile: 3.628

Compare this with your WinBUGS posterior
actual result.
Example 2
Lets look at an example where the parameter of interest is multivariate.

Let Y~N(mu, tau) where both mu and tau are unknown. In WinBUGS,
tau=1/variance is the way WinBUGS specifies the Normal
distribution.

We know tau is is positive, so we pick a gamma prior for sigma.

Lets assume mu is known to be in the range 0 to 10, but that lower
values are more likely to occur. So we pick an exponential prior for
mu, and restrict possible values to be <10.
Example 2 Code
#STATE THE LIKELIHOOD

Model
{
for (i in 1:20)
{
y[i]~dnorm(mu,tau)
}
#PRIOR DISTRIBUTIONS FOR TAU&MU
#Note: TAU is 1/Variance, also called the precision

tau~dgamma(a,b)
mu~dexp(1)I(0,10)
var<-1/tau
sigma<-sqrt(var)
}

#ENTER THE DATA
list(y=c(2.00161, 4.8484, 2.94069, 1.67246, -1.08507, 2.20847, 1.43717, 4.33718, 0.47811, 1.83598, 1.4771, 2.37736, 0.79921, 3.70158,
0.3985, 3.12491, -3.31308, 1.93582, 2.04038, 0.25967),a=1,b=2)

#SET INITIAL VALUES
list(mu=5,tau=2)
list(mu=1,tau=5)
list(mu=9,tau=8)

Example 2
The data came from a N(1.6, Var=4)
distribution. Do your posterior distributions
support this?

Try changing some of the parameters of
your prior distributions and re-running the
code and see what happens to your
estimates.
Example 3 Logistic Regression
Researchers were interested in p, the probability
of germination of 2 types of seeds, planted
according to 2 root extract methods.
To get the code, go to the Help menu in
WinBUGS. Click Examples Vol I.
Then click the Seeds: random effects logistic
regression link.
Authors used non-informative priors (high
variance).
Run the code to get posterior estimates for
alpha0, alpha1, alpha2, and alpha12.
Burn In Samples
Often users use a burn in period of initial
samples that they discard when estimating
the posterior distribution. This period
allows the MCMC sampling procedure to
stabilize.
To leave out k burn-in samples for
estimating the posterior, enter the number
k+1 in the Beg field of the Sample
Monitor Tool.
Final Points
The Help manual contains much useful information,
including several examples.
You should Always verify how WinBUGS specifies the
distributions. For example, their scale parameter for the
normal distribution is the inverse of
WinBUGS can be obtained for FREE from the following
website:
http://www.mrc-bsu.cam.ac.uk/bugs/
Click the WinBUGS option on the left side of the screen
To learn more, take Dr. Ghoshs Bayesian Inference
class offered this fall.

2
o