REASONING UNDER UNCERTAINTY 

In most real domains there is much knowledge that cannot 
be known absolutely.

Example: 
	In planning a trip to a meeting in Houston, if I leave 
	College Station 2 hours early will I make it on time.

There are too many variables (construction, traffic, police, 
chance of car breaking down, etc.) to reason about this 
absolutely.

Instead our systems can encode likelihood or belief that 
events will happen.

	P(CS -> Houston in 2 hours) = .8

How do we reason using such beliefs?

Consider your own assessment:

A patient comes into the office suffering from severe 
stomach cramps and worried that it is food poisoning. 
Given that 99% of food poisoning cases include severe 
stomach cramps, 1% of people without food poisoning 
experience severe stomach cramps, and .01% of all 
patients suffers from food poisoning, what is your 
estimate for how likely food poisoning is the cause?



				BASIC PROBABILITY

Unconditional (Prior) probabilities

The probability that something will occur without any other 
knowledge:
	P(sunny) = .50
	P(cloudy) = .50
	P(rainy) = .15
	P(snowy) = .01

Conditional probabilities

The probability that something will occur based on existing 
knowledge:
	P(rainy | cloudy) = .3
	P(cavity | toothache) = .8

Conditional probabilities can be stated in terms of 
prior probabilities
	P(A | B) = P(A ^ B) / P(B) for P(B) > 0
	P(A ^ B) = P(B) * P(A | B)
also
	P(A ^ B) = P(A) * P(B | A)



			AXIOMS OF PROBABILITY

All probabilities are between 0 and 1
	0 <= P(A) <= 1

Valid statements have probability 1 and unsatisfiable 
statements have a probability 0.
	P(True) = 1, P(False) = 0

Probability of a disjunction is given by:
	P(A v B) = P(A) + P(B) - P(A^ B)

Venn Diagram Perspective



			JOINT PROBABILITY DISTRIBUTION

have probabilistic knowledge of all subsets within state space

			Toothache	not Toothache
Cavity			    .04		    .06
not Cavity		    .01		    .89

P(Cavity|Toothache) = P(Cavity^Toothache) / P(Toothache)
		    = .04 / (.04 + .01)
		    = .8

For N variables, the joint probability matrix will include 2N 
entries.

For real-world problems with hundreds or thousands of 
variables, this is not of practical use.

Need a way to work with probabilities without having 
complete knowledge of all at the start.



				BAYES' RULE

Deriving Bayes' Rule

Remember:
	P(A ^ B) = P(A | B) * P(B)
	P(A ^ B) = P(B | A) * P(A)

Setting equal and dividing by P(A) yields Bayes' Rule:
	P(B | A) = P(A | B) * P(B) / P(A)


Applying Bayes' Rule

Given the following facts:
	prior probability of patient having meningitis is 1/50,000
	prior probability of patient having stiff neck is 1/20
	meningitis causes stiff neck in 50% of cases

What is the probability of a patient with a stiff neck having 
meningitis?

	P(M | S) = P(S | M) * P(M) / P(S)
		 = (.5 * 1/50000) / (1/20)
		 = .0002



				NORMALIZATION

What if there was an outbreak of meningitis in town?

How can our system make decisions when there are 
unknown changes in prior probabilities.

Normalize to remove prior probabilities:
	P(M | S) = P(S | M) * P(M) / P(S)
	P(not M | S) = P(S | not M) * P(not M) / P(S)
	P(M | S) + P(not M | S) = 1
	P(S) = P(S | M) * P(M) + P(S | not M) * P(not M)

Substitute for P(S) in equation for P(M | S) yields:
	P(M | S) = (P(S | M) * P(M)) / 
		   (P(S | M) * P(M) + P(S | not M) * P(not M))

Another way of reading this is that:
	P(M | S) = # of cases where S is caused by M /
		   total # of cases of S (caused by M or not)



			BAYES' RULE IN PRACTICE 

A patient comes into the office suffering from severe 
stomach cramps and worried that the cause is food 
poisoning. Given that 99% of food poisoning cases include 
severe stomach cramps, 1% of people without food 
poisoning experience severe stomach cramps, and .01% 
of all patients suffer from food poisoning, what is your 
estimate for how likely food poisoning is the cause?

Answering that original question:

What do we know:
	P(C | FP) = .99
	P(FP) = .0001
	P(C | not FP) = .01

Using normalized version of Bayes' Rule:
	pCases = P(C | FP) * P(FP) = .99 * .0001 = .000099
	nCases = P(C | not PF) * P(not FP) = .01 * .9999 = .009999
	P(FP | C) = pCases / (pCases + nCases) 
		  = .000099 / (.000099 + .009999)
		  = 1 / 102
		  < 1%



		COMBINING EVIDENCE AND CONDITIONAL INDEPENDENCE

How do we deal with multiple pieces of evidence?

If we had the entire joint probability distribution, it would be 
easy ... but we don't.

With two pieces of evidence (B and C):
	P(A | B ^ C) = P(B ^ C | A) * P(A) / P(B ^ C)

Rewriting Bayes' Rule:
	P(A | B) = P(A) * P(B | A) / P(B)

with some more rewriting ...
	P(A | B ^ C) = P(A | B) * P(C | A ^ B) / P(C | B)
		     = P(A) * (P(B | A) / P(B)) * P(C | A ^ B) / P(C | B)

Now we have to make some assumptions to progress. If 
we assume that symptoms are conditionally independent 
(not the cause of one another) we can simplify this.

As an exercise left for the reader ...



			DISCUSSION

How could we acquire such beliefs, probablities in practice?
	surveys, questionaires, observations of practice

Observations show humans don't follow Baysian inference -- why
might this be?
	belief values are rough (not exact probabilities)
	reasoning about beliefs is not an exact process

Another approach to uncertain reasoning is to watch how experts
reason about their beliefs and build systems which mimic this.