Reasoning with Bayesian Belief Networks
Reasoning
with Bayesian
Belief Networks
bbn.pptx
Overview
•
Bayesian Belief Networks (BBNs) can reason
with networks of propositions and associated
probabilities
•
BBNs encode causal associations between
facts and events the propositions represent
•
Useful for many AI problems
–
Diagnosis
–
Expert systems
–
Planning
–
Learning
Judea Pearl
•
UCLA CS professor
•
Introduced
Bayesian
networks
in the 1980
•
Pioneer of probabilistic
approach to AI reasoning
•
First to mathematize causal
modeling in empirical sciences
•
Written many books on the
topics, including the popular
2018
Book of Why
BBN Definition
•
AKA Bayesian Network, Bayes Net
•
A graphical model (as a DAG) of probabilistic
relationships among a set of random variables
•
Nodes are variables, links represent direct
influence of one variable on another
•
Nodes have prior
probabilities or
Conditional
Probability
Tables (CPTs)
source
Recall Bayes Rule
N
o
t
e
s
y
m
m
e
t
r
y
:
c
a
n
c
o
m
p
u
t
e
p
r
o
b
a
b
i
l
i
t
y
o
f
a
h
y
p
o
t
h
e
s
i
s
g
i
v
e
n
i
t
s
e
v
i
d
e
n
c
e
a
s
w
e
l
l
a
s
p
r
o
b
a
b
i
l
i
t
y
o
f
e
v
i
d
e
n
c
e
g
i
v
e
n
h
y
p
o
t
h
e
s
i
s
Simple Bayesian Network
Cancer
Smoking
Simple Bayesian Network
Cancer
Smoking
Nodes
represent
variables
•
S
m
o
k
i
n
g
v
a
r
i
a
b
l
e
r
e
p
r
e
s
e
n
t
s
p
e
r
s
o
n
’
s
d
e
g
r
e
e
o
f
s
m
o
k
i
n
g
a
n
d
h
a
s
t
h
r
e
e
p
o
s
s
i
b
l
e
v
a
l
u
e
s
(
n
o
,
l
i
g
h
t
,
h
e
a
v
y
)
•
C
a
n
c
e
r
v
a
r
i
a
b
l
e
r
e
p
r
e
s
e
n
t
s
p
e
r
s
o
n
’
s
c
a
n
c
e
r
d
i
a
g
n
o
s
i
s
a
n
d
h
a
s
t
h
r
e
e
p
o
s
s
i
b
l
e
v
a
l
u
e
s
(
n
o
n
e
,
b
e
n
i
g
n
,
m
a
l
i
g
n
a
n
t
)
Simple Bayesian Network
Cancer
Smoking
Directed links
represent
“
causal
”
relations
•
t
l
;
d
r
:
s
m
o
k
i
n
g
e
f
f
e
c
t
s
c
a
n
c
e
r
•
S
m
o
k
i
n
g
b
e
h
a
v
i
o
r
e
f
f
e
c
t
s
t
h
e
p
r
o
b
a
b
i
l
i
t
y
o
f
c
a
n
c
e
r
o
u
t
c
o
m
e
•
S
m
o
k
i
n
g
b
e
h
a
v
i
o
r
c
o
n
s
i
d
e
r
e
d
e
v
i
d
e
n
c
e
f
o
r
w
h
e
t
h
e
r
a
p
e
r
s
o
n
i
s
l
i
k
e
l
y
t
o
h
a
v
e
c
a
n
c
e
r
o
r
n
o
t
Simple Bayesian Network
Cancer
Smoking
P
r
i
o
r
p
r
o
b
a
b
i
l
i
t
y
o
f
S
J
o
i
n
t
d
i
s
t
r
i
b
u
t
i
o
n
o
f
S
a
n
d
C
N
o
d
e
s
w
i
t
h
o
u
t
i
n
-
l
i
n
k
s
h
a
v
e
p
r
i
o
r
p
r
o
b
a
b
i
l
i
t
i
e
s
N
o
d
e
s
w
i
t
h
i
n
-
l
i
n
k
s
h
a
v
e
j
o
i
n
t
p
r
o
b
a
b
i
l
i
t
y
d
i
s
t
r
i
b
u
t
i
o
n
s
More Complex Bayesian Network
Smoking
Gender
Age
Cancer
Lung
Tumor
Serum
Calcium
Exposure
to Toxics
More Complex Bayesian Network
Smoking
Gender
Age
Cancer
Lung
Tumor
Serum
Calcium
Exposure
to Toxics
Links represent
immediate
“
causal
”
relations
Nodes
represent
variables
•
Does gender
cause smoking?
•
Influence might
be a better term
More Complex Bayesian Network
Smoking
Gender
Age
Cancer
Lung
Tumor
Serum
Calcium
Exposure
to Toxics
condition
More Complex Bayesian Network
Smoking
Gender
Age
Cancer
Lung
Tumor
Serum
Calcium
Exposure
to Toxics
predispositions
More Complex Bayesian Network
Smoking
Gender
Age
Cancer
Lung
Tumor
Serum
Calcium
Exposure
to Toxics
observable symptoms
More Complex Bayesian Network
Smoking
Gender
Age
Cancer
Lung
Tumor
Serum
Calcium
Exposure
to Toxics
•
Model has 7
variables
•
Complete joint
probability
distribution will
have 7
dimensions!
•
Too much data
required
•
BBN simplifies: a
node has a CPT
with data on
itself & parents in
graph
C
a
n
w
e
p
r
e
d
i
c
t
l
i
k
e
l
i
h
o
o
d
o
f
l
u
n
g
t
u
m
o
r
g
i
v
e
n
v
a
l
u
e
s
o
f
o
t
h
e
r
6
v
a
r
i
a
b
l
e
s
?
Independence
Age
and
Gender
are
independent.
P(A |G) = P(A)
P(G |A) = P(G)
Gender
Age
P(A,G) = P(G|A) P(A) = P(G)P(A)
P(A,G) = P(A|G) P(G) = P(A)P(G)
P(A,G) = P(G) * P(A)
There is no path
between them in
the graph
Conditional Independence
Smoking
Gender
Age
Cancer
Cancer
is independent
of
Age
and
Gender
given
Smoking
P(C | A,G,S) = P(C | S)
If we know value of smoking, no need
to know values of age or gender
Conditional Independence
Smoking
Gender
Age
Cancer
Cancer
is independent
of
Age
and
Gender
given
Smoking
•
Instead of one big CPT with 4
variables, we have two smaller
CPTs with 3 and 2 variables
•
If all variables binary: 12 models
(2
3
+2
2
) rather than 16 (2
4
)
Conditional Independence: Naïve Bayes
Cancer
Lung
Tumor
Serum
Calcium
Serum Calcium
and
Lung
Tumor
are dependent
Naïve Bayes
assumption: evidence (e.g., symptoms)
independent given disease; easy to combine evidence
Explaining Away
Exposure to Toxics
is
dependent
on
Smoking
, given
Cancer
Exposure to Toxics
and
Smoking
are independent
Smoking
Cancer
Exposure
to Toxics
•
Explaining away:
reasoning pattern where confirma-
tion of one cause reduces need to invoke alternatives
•
Essence of
Occam’s Razor
(prefer hypothesis with
fewest assumptions)
•
Relies on independence of causes
P(E=heavy | C=malignant) > P(E=heavy
| C=malignant, S=heavy)
Conditional Independence
Smoking
Gender
Age
Cancer
Lung
Tumor
Serum
Calcium
Exposure
to Toxics
Cancer
is independent
of
Age
and
Gender
given
Exposure to
Toxics
and
Smoking
.
Descendants
Parents
Non-Descendants
A variable (node) is conditionally independent
of its non-descendants given its parents
Another non-descendant
Diet
Cancer
is independent
of
Diet
given
Exposure
to
Toxics
and
Smoking
Smoking
Gender
Age
Cancer
Lung
Tumor
Serum
Calcium
Exposure
to Toxics
A variable is
conditionally
independent of its
non-descendants
given its parents
BBN Construction
The
knowledge acquisition
process for a BBN
involves three steps
KA1
: Choosing appropriate variables
KA2
: Deciding on the network structure
KA3
: Obtaining data for the conditional
probability tables
•
They should be values, not probabilities
KA1: Choosing variables
•
Variable values: integers, reals or enumerations
•
Variable should have collectively
exhaustive
,
mutually exclusive
values
Heuristic: Knowable in Principle
Example of good variables
–
Weather: {Sunny, Cloudy, Rain, Snow}–
Gasoline: $ per gallon {<1, 1-2, 2-3, 3-4, >4}–
Temperature: {
100
°
F , < 100
°
F}
–
User needs help on Excel Charts: {Yes, No}–
User’
s personality: {dominant, submissive}KA2: Structuring
Lung
Tumor
Network structure corresponding
to
“
causality
”
is usually good.
Initially this uses designer’s
knowledge and intuitions but
can be checked with data
May be better to add
suspected links than to
leave out
But bigger CPT tables mean
more data collection
KA3: The Numbers
•
F
o
r
e
a
c
h
v
a
r
i
a
b
l
e
w
e
h
a
v
e
a
t
a
b
l
e
o
f
p
r
o
b
a
b
i
l
i
t
y
o
f
i
t
s
v
a
l
u
e
f
o
r
v
a
l
u
e
s
o
f
i
t
s
p
a
r
e
n
t
s
•
F
o
r
v
a
r
i
a
b
l
e
s
w
/
o
p
a
r
e
n
t
s
,
w
e
h
a
v
e
p
r
i
o
r
p
r
o
b
a
b
i
l
i
t
i
e
s
KA3: The numbers
•
Zeros and ones are often enough
•
Order of magnitude is typical: 10
-9
vs 10
-6
•
Sensitivity analysis can be used to decide
accuracy needed
•
Second decimal usually doesn’
t matter
•
Relative probabilities are important
Three kinds of reasoning
BBNs support three main kinds of reasoning:
•
Predicting
conditions given predispositions
•
Diagnosing
conditions given symptoms (and
predisposing)
•
Explaining
a condition by one or more
predispositions
To which we can add a fourth:
•
Deciding
on an action based on probabilities
of the conditions
Predictive Inference
How likely are
elderly males
to get
malignant cancer
?
P(
C=malignant
|
Age>60, Gender=male
)
Smoking
Gender
Age
Cancer
Lung
Tumor
Serum
Calcium
Exposure
to Toxics
Predictive and diagnostic combined
How likely is an
elderly
male
patient with high
Serum Calcium
to have
malignant cancer
?
P(
C=malignant
|
Age>60,
Gender= male, Serum Calcium = high
)
Smoking
Gender
Age
Cancer
Lung
Tumor
Serum
Calcium
Exposure
to Toxics
Explaining away
Smoking
Gender
Age
Cancer
Lung
Tumor
Serum
Calcium
Exposure
to Toxics
•
If we see a
lung tumor
, the
probability of
heavy
smoking
and of
exposure
to toxics
both go up
Some software tools
•
Netica
: Windows app for working with Bayes-
ian belief networks and influence diagrams
–
A commercial product, free for small networks
–
Includes graphical editor, compiler, inference
engine, etc.
–
To run in OS X or Linus you need Crossover
•
Hugin
: free demo versions for Linux, Mac, and
Windows are available
•
Various Python packages, e.g., …
•
Aima-python code in probability4e.py
Dyspnea is difficult or
labored breathing
Same BBN model in Hugin app
Decision making
•
A decision is a medical domain might be a
choice of treatment (e.g., radiation or
chemotherapy)
•
Decisions should be made to maximize
expected utility
•
View decision making in terms of
–
Beliefs/Uncertainties
–
Alternatives/Decisions
–
Objectives/Utilities
Decision Problem
Should I have my party
inside or outside?
Value Function
A numerical score over all possible states allows
a BBN to be used to make decisions
Using $ for the value
helps our intuition
Decision Making with BBNs
•
Today’s weather forecast might be either
sunny, cloudy or rainy
•
Should you take an umbrella when you leave?
•
Your decision depends only on the forecast
–
The forecast “depends on” the actual weather
•
Your satisfaction depends on your decision and
the weather
–
Assign a utility to each of four situations: (rain|no
rain) x (umbrella, no umbrella)
Decision Making with BBNs
•
Extend BBN framework to include two new
kinds of nodes:
decision
and
utility
•
Decision
node computes the expected utility
of a decision given its parent(s) (e.g., forecast)
and a valuation
•
Utility
node computes utility value given its
parents, e.g. a decision and weather
•
Assign utility to each situations: (rain|no rain) x
(umbrella, no umbrella)
•
Utility value assigned to each is probably subjective
P
r
e
d
i
s
p
o
s
i
t
i
o
n
s
o
r
c
a
u
s
e
s
C
o
n
d
i
t
i
o
n
s
o
r
d
i
s
e
a
s
e
s
F
u
n
c
t
i
o
n
a
l
N
o
d
e
S
y
m
p
t
o
m
s
o
r
e
f
f
e
c
t
s
Dyspnea is
shortness of
breath
Bayesian Belief Networks (BBNs) provide a powerful framework for reasoning with probabilistic relationships between variables. Introduced by Judea Pearl in the 1980s, BBNs encode causal associations and are used in various AI applications such as diagnosis, expert systems, planning, and learning. The graphical model of BBNs, with nodes representing variables and links indicating influences, allows for efficient probabilistic reasoning and decision-making. By applying the Bayes Rule, BBNs enable the calculation of probabilities based on evidence, making them valuable tools in understanding complex systems.
Download Presentation
Please find below an Image/Link to download the presentation.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author. Download presentation by click this link. If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.
E N D
Presentation Transcript
bbn.pptx Reasoning with Bayesian Belief Networks
Overview Bayesian Belief Networks (BBNs) can reason with networks of propositions and associated probabilities BBNs encode causal associations between facts and events the propositions represent Useful for many AI problems Diagnosis Expert systems Planning Learning
Judea Pearl UCLA CS professor Introduced Bayesian networks in the 1980 Pioneer of probabilistic approach to AI reasoning First to mathematize causal modeling in empirical sciences Written many books on the topics, including the popular 2018 Book of Why
BBN Definition AKA Bayesian Network, Bayes Net A graphical model (as a DAG) of probabilistic relationships among a set of random variables Nodes are variables, links represent direct influence of one variable on another Nodes have prior probabilities or Conditional Probability Tables (CPTs) source
Recall Bayes Rule ) | ( ) P E H P = = ( , ( ) ( | ) ( ) P H E E P E H P H ( | P ) ( ) P E H P H = ( | ) P H E ( ) E Note symmetry: can compute probability of a hypothesis given its evidence as well as probability of evidence given hypothesis
Simple Bayesian Network , , S no light heavy Smoking Cancer , , C none benign malignant
Simple Bayesian Network , , S no light heavy Smoking Cancer Nodes represent variables , , C none benign malignant Smokingvariable represents person s degree of smoking and has three possible values (no, light, heavy) Cancervariable represents person s cancer diagnosis and has three possible values (none, benign, malignant)
Simple Bayesian Network , , S no light heavy Smoking Cancer , , C none benign malignant tl;dr: smoking effects cancer Smoking behavior effects the probability of cancer outcome Smoking behavior considered evidence for whether a person is likely to have cancer or not Directed links represent causal relations
Simple Bayesian Network , , S no light heavy Smoking Cancer , , C none benign malignant Prior probability of S P(S=no) P(S=light) P(S=heavy) 0.05 0.80 0.15 Nodes without in-links have prior probabilities Joint distribution of S and C Smoking= no C=none C=benign C=malignant 0.01 0.04 light heavy 0.60 0.25 0.15 Nodes with in-links have joint probability distributions 0.96 0.88 0.03 0.08
More Complex Bayesian Network Age Gender Exposure to Toxics Smoking Cancer Serum Calcium Lung Tumor
More Complex Bayesian Network Nodes represent variables Age Gender Exposure to Toxics Smoking Does gender cause smoking? Links represent immediate causal relations Cancer Influence might be a better term Serum Calcium Lung Tumor
More Complex Bayesian Network Age Gender Exposure to Toxics Smoking condition Cancer Serum Calcium Lung Tumor
More Complex Bayesian Network predispositions Age Gender Exposure to Toxics Smoking Cancer Serum Calcium Lung Tumor
More Complex Bayesian Network Age Gender Exposure to Toxics Smoking Cancer observable symptoms Serum Calcium Lung Tumor
More Complex Bayesian Network Model has 7 variables Complete joint probability distribution will have 7 dimensions! Too much data required BBN simplifies: a node has a CPT with data on itself & parents in graph Age Gender Can we predict likelihood of lung tumor given values of other 6 variables? Exposure to Toxics Smoking Cancer Serum Calcium Lung Tumor
Independence Age and Gender are independent. Age Gender P(A,G) = P(G) * P(A) There is no path between them in the graph P(A |G) = P(A) P(G |A) = P(G) P(A,G) = P(G|A) P(A) = P(G)P(A) P(A,G) = P(A|G) P(G) = P(A)P(G)
Conditional Independence Cancer is independent of Age and Gender given Smoking Age Gender Smoking P(C | A,G,S) = P(C | S) Cancer If we know value of smoking, no need to know values of age or gender
Conditional Independence Cancer is independent of Age and Gender given Smoking Age Gender Instead of one big CPT with 4 variables, we have two smaller CPTs with 3 and 2 variables Smoking If all variables binary: 12 models (23 +22) rather than 16 (24) Cancer
Conditional Independence: Nave Bayes Serum Calcium and Lung Tumor are dependent Cancer Serum Calcium is indepen- dent of Lung Tumor, given Cancer Serum Calcium Lung Tumor P(L | SC,C) = P(L|C) P(SC | L,C) = P(SC|C) Na ve Bayes assumption: evidence (e.g., symptoms) independent given disease; easy to combine evidence
Explaining Away Exposure to Toxics and Smoking are independent Exposure to Toxics Smoking Exposure to Toxics is dependent on Smoking, given Cancer Cancer P(E=heavy | C=malignant) > P(E=heavy | C=malignant, S=heavy) Explaining away: reasoning pattern where confirma- tion of one cause reduces need to invoke alternatives Essence of Occam s Razor (prefer hypothesis with fewest assumptions) Relies on independence of causes
Conditional Independence A variable (node) is conditionally independent of its non-descendants given its parents Age Gender Non-Descendants Exposure to Toxics Smoking Parents Cancer is independent of Age and Gender given Exposure to Toxics and Smoking. Cancer Serum Calcium Lung Tumor Descendants
Another non-descendant A variable is conditionally independent of its non-descendants given its parents Age Gender Exposure to Toxics Smoking Cancer Cancer is independent of Diet given Exposure to Toxics and Smoking Diet Serum Calcium Lung Tumor
BBN Construction The knowledge acquisition process for a BBN involves three steps KA1: Choosing appropriate variables KA2: Deciding on the network structure KA3: Obtaining data for the conditional probability tables
KA1: Choosing variables Variable values: integers, reals or enumerations Variable should have collectively exhaustive, mutually exclusive values Error Occurred x x x x 1 x 2 3 i 4 No Error ( ) x j i j They should be values, not probabilities Risk of Smoking Smoking
Heuristic: Knowable in Principle Example of good variables Weather: {Sunny, Cloudy, Rain, Snow} Gasoline: $ per gallon {<1, 1-2, 2-3, 3-4, >4} Temperature: { 100 F , < 100 F} User needs help on Excel Charts: {Yes, No} User s personality: {dominant, submissive}
KA2: Structuring Network structure corresponding to causality is usually good. Age Gender Initially this uses designer s knowledge and intuitions but can be checked with data Exposure to Toxic Smoking May be better to add suspected links than to leave out Genetic Damage Cancer Lung Tumor But bigger CPT tables mean more data collection
KA3: The Numbers For each variable we have a table of probability of its value for values of its parents For variables w/o parents, we have prior probabilities , , S C no none light , heavy Smoking Cancer , benign malignant smoking light smoking priors cancer no heavy no 0.80 none 0.96 0.88 0.60 light 0.15 benign 0.03 0.08 0.25 heavy 0.05 malignant 0.01 0.04 0.15
KA3: The numbers Second decimal usually doesn t matter Relative probabilities are important Zeros and ones are often enough Order of magnitude is typical: 10-9 vs 10-6 Sensitivity analysis can be used to decide accuracy needed
Three kinds of reasoning BBNs support three main kinds of reasoning: Predicting conditions given predispositions Diagnosing conditions given symptoms (and predisposing) Explaining a condition by one or more predispositions To which we can add a fourth: Deciding on an action based on probabilities of the conditions
Predictive Inference Age Gender How likely are elderly males to get malignant cancer? Exposure to Toxics Smoking P(C=malignant | Age>60, Gender=male) Cancer Serum Calcium Lung Tumor
Predictive and diagnostic combined How likely is an elderly male patient with high Serum Calcium to have malignant cancer? Age Gender Exposure to Toxics Smoking P(C=malignant | Age>60, Gender= male, Serum Calcium = high) Cancer Serum Calcium Lung Tumor
Explaining away Age Gender If we see a lung tumor, the probability of heavy smoking and of exposure to toxics both go up Exposure to Toxics Smoking Smoking If we then observe heavy smoking, the probability of exposure to toxics goes back down Cancer Serum Calcium Lung Tumor
Some software tools Netica: Windows app for working with Bayes- ian belief networks and influence diagrams A commercial product, free for small networks Includes graphical editor, compiler, inference engine, etc. To run in OS X or Linus you need Crossover Hugin: free demo versions for Linux, Mac, and Windows are available Various Python packages, e.g., Aima-python code in probability4e.py
Dyspnea is difficult or labored breathing
Decision making A decision is a medical domain might be a choice of treatment (e.g., radiation or chemotherapy) Decisions should be made to maximize expected utility View decision making in terms of Beliefs/Uncertainties Alternatives/Decisions Objectives/Utilities
Decision Problem Should I have my party inside or outside? dry Regret in wet Relieved dry Perfect! out wet Disaster
Value Function A numerical score over all possible states allows a BBN to be used to make decisions Location? Weather? in in out out Value $50 $60 $100 $0 dry wet dry wet Using $ for the value helps our intuition
Decision Making with BBNs Today s weather forecast might be either sunny, cloudy or rainy Should you take an umbrella when you leave? Your decision depends only on the forecast The forecast depends on the actual weather Your satisfaction depends on your decision and the weather Assign a utility to each of four situations: (rain|no rain) x (umbrella, no umbrella)
Decision Making with BBNs Extend BBN framework to include two new kinds of nodes: decision and utility Decision node computes the expected utility of a decision given its parent(s) (e.g., forecast) and a valuation Utility node computes utility value given its parents, e.g. a decision and weather Assign utility to each situations: (rain|no rain) x (umbrella, no umbrella) Utility value assigned to each is probably subjective
Symptoms or effects Dyspnea is shortness of breath
