Artificial Intelligence

Wester Zela
Week Six
edzela@yahoo.com
www.ssyspe.org
Agenda

• Bayesian Network
• Examples: Cervical Spondylotic Myelophaty (CSM)
• Test
• Bayesian Network in Weka
• Bayesian Network in Rstudio
• Home Work
Bayesian Network
• It can be referred as probabilistic directed acyclic graphical (DAG)
model, Bayes Network, Belief Network, or Bayesian Model.
• Based on a set of variables connected, it’s possible to predict
outcomes based on probabilities. The resulting value of one variable
will influence the output probability of another, hence the use of
networked nodes.
• A Bayesian Network manages to combine probability theory with
graph theory and provides a very handy method for dealing with
complexity and uncertainty.
A little graph theory
• Nodes and edges are connected.

Directed acyclic graphs (DAG)

A Little probability theory
• Coin Flip:
• P(Coin toss will be heads) = 0.5 (50%)
• P(Coin toss will be tails)=0.5 (50%)
• What is the probability of two heads? p(heads)x P(heads)=0.5x0.5=0.25 or
p(Heads first time | Heads second time)
• Lottery:
• What is probability to win the Tinka? :
p(Tinka) = 6/45x5/44x4/43x3/42x2/41x1/40=720/5,864,443,200
Bayes Theorem
• The important part of Bayes’ Theorem is the observation of previous
events, or your degree of belief that something will occur.
• If you have a degree of belief of one event happening, you can apply
it to new data and make an informed calculation to its probability.
Example:
C : Likes a Rock
¬C : Does not Likes a Rock
T : Test positive for liking Rock
¬T : Test Negative for liking Rock
P(T|C) = 0.95 P(¬T|¬C) = 0.99 P(C) = 0.02
Bayes Theorem
• Formula:

• Calculating:

• There is 51% that test will return true positive.

An example:
• Questions like: Is my backyard wet?
• Three nodes: Rain, YardWet, Hose
• Two events that cause that yard is wet
• Y = Yard wet (True or False)
• R = Raining (True or False)
• H = Someone using the hose (True or False)

p(Y,R,H) = p(Y|R,H) p(H|R) p(R)

What is the probability that it is raining when the yard is wet?
Assigning Probabilities
• Assigning probabilities to the graph
Calculating Results
• Given that the yard is wet, what is the probability that it is raining?
• So, YardWet(True) and Raining(True), Hose? (True or False).
• Probability:
• P(R=T|Y=T) = P(Y=T|H=T, R=T)P(H=T|R=T)P(R=T) + P(Y=T|H=F, R=T)P(H=F|R=T)P(R=T)
-----------------------------------------------------------------------------------------------
P(Y=T|H=T, R=T)P(H=T|R=T)P(R=T) + P(Y=T|H=F, R=T)P(H=F|R=T)P(R=T) +
P(Y=T|H=T, R=T)P(H=T|R=F)P(R=F) + P(Y=T|H=F, R=F)P(H=F|R=F)P(R=F)

• P(R=T|Y=F) = ?
Calculating results
• From a probability:
• When Hose = True
• p(Y=True, H=True, R=True)
• p(Y=True|H=True, R=True) x p(H=True|R=True) x p(R=True)
• 0.99 (Y=T, H=T, R=T) x 0.01 (H=T, R=T) x 0.2 (R=T) = 0.00198
• When Hose = False
• p(Y=True, H=False, R=True)
• p(Y=True|H=False, R=True) x p(H=False|R=True) x p(R=True)
• 0.8 (Y=T, H=F, R=T) x 0.99 (H=F, R=T) x 0.2 (R=T) = 0.1584
Calculating Results
• T,T,T is 0.00198
• TFT is 0.1584
• T,T,F is 0.9 x 0.4 x 0.8 = 0.288
• T,F,F is 0.0 x 0.6 x 0.8 = 0.0

• Finally the probability is = 0.3576 = 35.7%

Using Domain Experts
• Who Provide the probabilities of the bayesian network?
Example: Cervical Spondylotic Myelophaty
(CSM)
• Cervical Spondylotic Myelopathy (CSM) is a spine disease and is one
of the most common spinal cord dysfunctions in patients over 55
years old. Symptoms include impaired gait leading to issues walking,
and weakness in general. Surgical procedures are available and
performed when the CSM progresses to mild or severe symptoms.
• Nodes:
• Age of Patient (A)
• Does the patient smoke (S)
• Duration of symptoms (D)
• Surgical outcome success (SS)
Assigning probabilities
CSM graph

Calculate: P(A <55 | SS = Positive) ?

Java API for Bayesian Network
• Download API from https://github.com/jasebell/JavaBayesAPI

Build and Run the model:

• Open Eclipse y create the project BayesNetExample
• Add External JAR downloaded
• Copy the code give in Class and run the model.
Test
• Create a Bayesian Network for your final work
Learning Bayesian Network from Data:
Algorithm K2
• K2 is an algorithm for constructing Bayes Network from Data
• El problema que soluciona K2 es el de encontrar una red bayesiana
(modelo) a partir de una base de datos D, que contiene un set de
variables denominado Z, Bsi y Bsj que son dos modelos de red
bayesiana que contienen exactamente las mismas variables que están
en Z.

• By computing such ratios for pairs of Bayes network structures, we

can rank order a set of structures by their posterior probabilities.
Algorithm K2
n qi
(ri  1)! ri
P( Bs , D)  P( Bs ) Nij  Nijk !
i 1 j 1 ( N ij  ri  1)! k 1

• D - dataset, it has m cases(records)

• Z - a set of n discrete variables: (x1, …, xn)
• ri - a variable xi in Z has ri possible value assignment:
• Bs - a bayes network structure containing just the variables in Z
• i - each variable xi in Bs has a set of parents which we represent with a list of variables i
• qi - there are has unique instantiations of i
• wij - denote jth unique instantiation of i relative to D.
• Nijk - the number of cases in D in which variable xi has the value of vik
and i is instantiated as wij.
• Nij - ri
N ij   N ijk
k 1
Algorithm K2
Input: A set of n nodes, an ordering on the nodes, an upper bound u on the
number of parents a node may have, and a database D containing m cases.
Output: For each nodes, a printout of the parents of the node
Steps:
• Step 1: Grafo inicial sin arcos.
• Step 2: Elección de un arco para añadir al grafo.
• Step 2.1: Calcula la probabilidad de la nueva red con un arco nuevo en cada caso.
• Step 2.2: Escoge el arco que da probabilidad mayor.
• Step 3: Si el arco nuevo aumenta la probabilidad de la nueva red, se añade y se va
al paso 2, en caso contrario Fin, esa es la red.
Classifier Bayes Net.K2
Setting parameters
• initAsNaiveBayes: Cuando se establece en True será una red con una
flecha desde el nodo de clasificador a cada nodo. Cuando se
establece en falso, la estructura inicial corresponderá a una red vacía.
• markovBlanketClassier: Cuando se establece en true se asegura que
todos los nodos de la red forman parte de la manta de Markov
(Markov Blanket) del nodo de clasificador.
• maxNrOfParents: Establece el número máximo de padres que un
nodo podrá tener en la red bayesiana. Cuando se establece en 2, será
una red de bayes aumentada a árbol (TAN), y cuando se establece >2,
una red de bayes aumentada de red (BAN).
Setting parameters
• randomOrder: Cuando se establece en true, el orden de los nodos de
la red es al azar. El valor predeterminado es false y se utiliza el orden
que traen los nodos en el conjunto de datos.
• scoreType: El tipo de puntuación determina la medida utilizada para
evaluar la calidad de una estructura de red.
R Studio
• Install R
• https://cran.r-project.org/mirrors.html
• https://www.rstudio.com/products/rstudio/download/
• Install the bnlearn package
• Create some Bayesian Networks: http://www.bnlearn.com/examples/
Ranking in ssyspe.org: 28/04
• PTOi = 2NPi + 0.5NPAi
user_id Score 28/04 Ptos 28/04

1580 12.5 3
1565 12 3
1368 9 2
1563 4.5 1
1568 4.5 1
1571 4.5 1
1579 4.5 1
1566 2.5 0
1567 2.5 0
1569 2.5 0
1578 2.5 0
1564 2 0
1570 2 0
Paper of Week
• Predicting the prognosis of breast cancer by integrating clinical and
microarray data with Bayesian networks, Olivier Gevaert, Frank De
Smet, Dirk Timmerman, Yves Moreau and Bart De Moor;
BioInformatics.
References
• Machine Learning, Hands-On for Developers andTechnical Professionals, Jason
Bell, Willey.
• Machine Learning in Action, Peter Harrington, Manning Shelter Island
• Machine Learning Projects for .NET developers, Mathias Brandewinder, Apress