6.1 Assigning probabilities to events
6.2 Joint, marginal and conditional probability
6.3 Rules of probability
Lecture 5A:
LO1   Explain the importance of probability theory to statistical inference
LO2   Define the terms random experiment, sample space, elementary event, event, union of events, intersection of events, complement of an event and mutually exclusive events
LO3   Explain the different approaches to assigning probabilities to events
Lecture 5B:
LO4   Define the terms joint, marginal and conditional probabilities and independent events
LO5   Calculate probabilities using the basic probability rules – complement rule, addition rule and multiplication rule of probabilities
Random experiment
A random experiment is a process or course of action whose outcome is uncertain.
Examples
Experiment  Outcomes
Flip a coin  Heads and tails
Record statistics test marks  Numbers between 0 and 100
Measure the time to assemble  Numbers from 0 and above
a computer  
A list of all possible outcomes of a random experiment is called a sample space.

  Sample Space: S = {O1, O2, …, Ok}
There are three ways to assign a probability, P(Oi), to an outcome, Oi, namely:


If an experiment has n possible outcomes, this method would assign a probability of 1/n to each outcome. It is necessary to determine the number of possible outcomes.

Experiment 1:    Rolling a die
Outcomes      {1, 2, 3, 4, 5, 6}
Probabilities:    Each sample point has a 1/6 chance of occurring.

Experiment 2:  Rolling two dice and observing the total (X)
Outcomes:  {X} = {2, 3, …, 12}
Examples:


  P(X=2) = 1/36

  P(X=6) = 5/36
 
  P(X=10) = 3/36

Bits & Bytes Computer Shop tracks the number of desktop computer systems it sells over a month (30 days):

For example,
10 days out of 30 days
2 desktops were sold.

From this we can construct
the probabilities of an event
(i.e. the number of desktops sold on a given day)…
‘There is a 40% chance that 3 desktops will be sold on any given day.’
‘In the subjective approach we define probability as the degree of belief that we hold in the occurrence of an event.’

Example: Weather forecasting’s ‘P.O.P.’
‘Probability of Precipitation’ (or P.O.P.) is defined in different ways by different forecasters, but basically it’s a subjective probability based on past observations combined with current weather conditions.

POP 60% – based on current conditions, there is a 60% chance of rain (say).

No matter which method is used to assign probabilities all will be interpreted in the relative frequency approach.

i.e. the chance of occurrence
e.g.

Given a sample space S = {O1, O2, …, On}, the following characteristics for the probability P(Oi) of the simple event Oi must hold:
Probability of an event: the probability P(A) of event A is the sum of the probabilities assigned to the simple events contained in A.
We study methods to determine probabilities of events that result from combining other events in various ways.
There are several types of combinations and relationships between events:

  If A and B are two events, then
  P(A È B) = P(A occur or B occur or both)
  P(A Ç B) = P(A and B both occur)
  P(Ā) = P(A does not occur)
  P(A|B) = P(A occurs given that B has occurred)
For example, the rectangle stores all the possible tosses of 2 coins S = {(H,H), (H,T), (T,H), (T,T)}

Let A = observing at least one head
        = {(H,H), (H,T), (T,H)}
  Ac = {(T,T)}

P(A) = ¾ , P(Ac) = ¼
P(A) + P(Ac) = P(S) = 1
P(Ac) = 1 – P(A)
The intersection of events A and B is the set of all sample points that are in both A Ç B.

The intersection is denoted: A Ç B.

The joint probability of
A and B is the probability of
the intersection of A and B,
i.e. P(A Ç B).

For example, consider all possible tosses of two dice.
    S = {(1,1),(1,2),…,(6,6)}.
Let  A = tosses where first toss is 1
          = {(1,1),(1,2),(1,3),(1,4),(1,5),(1,6)}
and B = tosses where the second toss is 5
          = {(1,5),(2,5),(3,5),(4,5),(5,5),(6,5)}

The intersection, A Ç B = {(1,5)}.
The joint probability of A and B is 
The probability of the intersection
of A and B, i.e. P(A Ç B) = 1/36

The union of two events A and B, is the event containing all outcomes that are in A or B or both:

Union of A and B is denoted: A È B



For example, consider all possible tosses of two dice.
  S = {(1,1),(1,2), …,(6,6)}.
Let  A = tosses where first toss is 1
          = {(1,1),(1,2),(1,3),(1,4),(1,5),(1,6)}
and B = tosses where the second toss is 5
          = {(1,5),(2,5),(3,5),(4,5),(5,5),(6,5)}
  Union of A and B, A È B is
   A È B = {(1,1),(1,2),(1,3),(1,4),(1,5),        (1,6),(2,5),(3,5),(4,5),(5,5),(6,5)} (6,5)}



When two events are mutually exclusive (that is the two events have no outcomes in common), their joint probability is 0, hence:


The number of spots turning up when a 6-sided die is tossed is observed.
Consider the following events:
A: The number observed is at most 2 i.e. A={1,2}
B: The number observed is an even number. i.e. B={,2,4,6}
C: The number 4 turns up. i.e.  C={4}
Answer the following questions.


Solution
S = {1, 2, 3, 4, 5, 6}
Each simple event is equally likely to occur, thus,
P(1) = P(2) = … = P(6) = 1/6.
b) Find P(A).





Solution

c) Find P(Ac).





Solution

We have studied methods to determine probabilities of events that result from combining other events in various ways.
We have also learnt that there are several types of combinations and relationships between events:
The intersection of event A and B is the event that occurs when both A and B occur.
The intersection of events A and B is denoted by (A Ç B) or (A and B) .
The joint probability of A and B is the probability of the intersection of A and B, and is denoted by P(A Ç B).
A potential investor examined the relationship between the performance of mutual funds and the university where the fund manager earned his/her MBA. The relevant frequencies are given below
Dividing the frequencies by grand total gives
joint probabilities.
The joint probability of
[mutual fund outperforms …] and [… from a top 20 …] = 0.11.
P(A1 Ç B1) = 0.11

The joint probability of
[mutual fund outperforms …] and [… not from a top 20 …] = 0.06
  P(A2 Ç B1) = 0.06
Marginal probabilities are computed by adding across rows and down columns; that is they are calculated in the margins of the table.
These probabilities are computed by adding across rows and down columns.
Conditional probability is used to determine how two events are related; that is, we can determine the probability of one event given the occurrence of another related event.

Conditional probabilities are written as P(A|B) and read as ‘the probability of A given B’ and is calculated as:

 


Again, the probability of an event given that another event has occurred is called a conditional probability…

 






Find the conditional probability that a randomly selected fund is managed by a top 20 MBA program graduate, given that it did not outperform the market.

Solution:
A1 = Fund manager graduated from a top-20 MBA program
A2 = Fund manager did not graduate from a top-20 MBA program
B1 = Fund outperforms the market
B2 = Fund does not outperform the market
We need to find P(A1|B2).
 
We need to find P(A1|B2).
For event A1|B2, the new information that B2 is given reduces the relevant sample space for A1|B2 to 83% of event B2..

Before the new information becomes available, we have
  P(A1) = 0.40
After the new information on B2 becomes available, P(A1) changes to
  P(A1| B2) = 0.3494
Since the occurrence of B2 has changed the probability of A1, the two events are related (ie, A1 and B2 are not independent) and are called ‘dependent events’.
One of the objectives of calculating conditional probability is to determine whether two events are related.

In particular, we would like to know whether they are independent, that is, if the probability of one event is not affected by the occurrence of the other event.

Two events A and B are said to be independent if
  P(A|B) = P(A)
or
  P(B|A) = P(B)

Is ‘Fund outperforms the market’ dependent on the event that the ‘Manager graduated from a top-20 MBA program’?
Solution:
A1 = Fund manager graduated from a top-20 MBA program
A2 = Fund manager did not graduate from a top-20 MBA program
B1 = Fund outperforms the market
B2 = Fund does not outperform the market
In order to answer this question, it suffices to show that, for at least one i and j, P(Bi| Aj)¹ P(Bj)
If B2 and A2 are independent, then P(B2|A2)=P(B2)
We have already seen the dependency between A1 and B2, as P(A1|B2) = 0.3494 ¹ P(A1) = 0.40
Now, let us check the independence of A2 and B2.
P(B2) = 0.83
P(B2|A2)=P(B2 Ç A2)/P(A2) = 0.54/0.60 = 0.90
Therefore, P(B2|A2) ¹ P(B2)
Conclusion: A2 and B2 are dependent. That is, the probability of event B2 is affected by the occurrence of the event A2. This means that Fund outperforms the market does not depend on whether the manager graduated from a top-20-MBA program.
The personnel department of an insurance company has compiled data on promotion, classified by gender. Are promotion and gender dependent on each another?
If gender has no effect on promotion, then R and M are independent and P(R|M) = P(R). If this equality
holds, there is no difference in the probability of promotion between male and female managers.

As discussed earlier, the union event of A and B is the event that occurs when either A or B or both occur.
It is denoted (A È B) or (A or B).

Determine the probability that a randomly selected fund outperforms the market or the manager graduated from a top 20 MBA program.

That is, we want to calculate P (A1 È B1).
For any two event A and B
Complement rule:


Addition rule:

Multiplication Rule: 
General:
When A and B are independent

A stock market analyst feels that
The events of interest are:
A: The stock market rises
B: The company receives increased contribution
Calculate the following probabilities
  [at least moderate increase in earnings].
Solution
                 = 0.5 + 0.6 – 0.45 =0.65