Probability

Main Contents page | Index of all entries

An outcome is the result of an experiment or other situation involving uncertainty.

The set of all possible outcomes of a probability experiment is called a sample space.

The sample space is an exhaustive list of all the possible outcomes of an experiment. Each possible result of such a study is represented by one and only one point in the sample space, which is usually denoted by S.

Examples

Experiment Rolling a die once:

Sample space S = {1,2,3,4,5,6}

Experiment Tossing a coin:

Sample space S = {Heads,Tails}

Experiment Measuring the height (cms) of a girl on her first day at school:

Sample space S = the set of all possible real numbers

An event is any collection of outcomes of an experiment.

Formally, any subset of the sample space is an event.

Any event which consists of a single outcome in the sample space is called an elementary or simple event. Events which consist of more than one outcome are called compound events.

Set theory is used to represent relationships among events. In general, if A and B are two events in the sample space S, then

(A union B) = 'either A or B occurs or both occur'

(A intersection B) = 'both A and B occur'

(A is a subset of B) = 'if A occurs, so does B'

A' or = 'event A does not occur'

(the empty set) = an impossible event

S (the sample space) = an event that is certain to occur

Example

Experiment: rolling a dice once -

Sample space S = {1,2,3,4,5,6}

Events A = 'score < 4' = {1,2,3}

B = 'score is even' = {2,4,6}

C = 'score is 7' =

= 'the score is < 4 or even or both' = {1,2,3,4,6}

= 'the score is < 4 and even' = {2}

A' or = 'event A does not occur' = {4,5,6}

Relative frequency is another term for proportion; it is the value calculated by dividing the number of times an event occurs by the total number of times an experiment is carried out. The probability of an event can be thought of as its long-run relative frequency when the experiment is carried out many times.

If an experiment is repeated n times, and event E occurs r times, then the relative frequency of the event E is defined to be

rfn(E) = r/n

Example

Experiment: Tossing a fair coin 50 times (n = 50)

Event E = 'heads'

Result: 30 heads, 20 tails, so r = 30

Relative frequency: rfn(E) = r/n = 30/50 = 3/5 = 0.6

If an experiment is repeated many, many times without changing the experimental conditions, the relative frequency of any particular event will settle down to some value. The probability of the event can be defined as the limiting value of the relative frequency:

P(E) = rfn(E)

For example, in the above experiment, the relative frequency of the event 'heads' will settle down to a value of approximately 0.5 if the experiment is repeated many more times.

A probability provides a quantatative description of the likely occurrence of a particular event. Probability is conventionally expressed on a scale from 0 to 1; a rare event has a probability close to 0, a very common event has a probability close to 1.

The probability of an event has been defined as its long-run relative frequency. It has also been thought of as a personal degree of belief that a particular event will occur (subjective probability).

In some experiments, all outcomes are equally likely. For example if you were to choose one winner in a raffle from a hat, all raffle ticket holders are equally likely to win, that is, they have the same probability of their ticket being chosen. This is the equally-likely outcomes model and is defined to be:

P(E) =

number of outcomes corresponding to event E total number of outcomes

Examples

1. The probability of drawing a spade from a pack of 52 well-shuffled playing cards is 13/52 = 1/4 = 0.25 since

   event E = 'a spade is drawn';

   the number of outcomes corresponding to E = 13 (spades);

   the total number of outcomes = 52 (cards).

2. When tossing a coin, we assume that the results 'heads' or 'tails' each have equal probabilities of 0.5.

A subjective probability describes an individual's personal judgement about how likely a particular event is to occur. It is not based on any precise computation but is often a reasonable assessment by a knowledgeable person.

Like all probabilities, a subjective probability is conventionally expressed on a scale from 0 to 1; a rare event has a subjective probability close to 0, a very common event has a subjective probability close to 1.

A person's subjective probability of an event describes his/her degree of belief in the event.

Example
A Rangers supporter might say, "I believe that Rangers have probability of 0.9 of winning the Scottish Premier Division this year since they have been playing really well."

Two events are independent if the occurrence of one of the events gives us no information about whether or not the other event will occur; that is, the events have no influence on each other.

In probability theory we say that two events, A and B, are independent if the probability that they both occur is equal to the product of the probabilities of the two individual events, i.e.

The idea of independence can be extended to more than two events. For example, A, B and C are independent if:

1. A and B are independent; A and C are independent and B and C are independent (pairwise independence);
2. 

If two events are independent then they cannot be mutually exclusive (disjoint) and vice versa.

Example

Suppose that a man and a woman each have a pack of 52 playing cards. Each draws a card from his/her pack. Find the probability that they each draw the ace of clubs.

We define the events:

A = probability that man draws ace of clubs = 1/52

B = probability that woman draws ace of clubs = 1/52

Clearly events A and B are independent so:

= 1/52 . 1/52 = 0.00037

That is, there is a very small chance that the man and the woman will both draw the ace of clubs.

See also conditional probability.

Two events are mutually exclusive (or disjoint) if it is impossible for them to occur together.

Formally, two events A and B are mutually exclusive if and only if

If two events are mutually exclusive, they cannot be independent and vice versa.

Examples

1. Experiment: Rolling a die once

   Sample space S = {1,2,3,4,5,6}

   Events A = 'observe an odd number' = {1,3,5}

   B = 'observe an even number' = {2,4,6}

   = the empty set, so A and B are mutually exclusive.

   
   
2. A subject in a study cannot be both male and female, nor can they be aged 20 and 30. A subject could however be both male and 20, or both female and 30.

The addition rule is a result used to determine the probability that event A or event B occurs or both occur.

The result is often written as follows, using set notation:

where:

P(A) = probability that event A occurs

P(B) = probability that event B occurs

= probability that event A or event B occurs

= probability that event A and event B both occur

For mutually exclusive events, that is events which cannot occur together:

= 0

The addition rule therefore reduces to

= P(A) + P(B)

For independent events, that is events which have no influence on each other:

The addition rule therefore reduces to

Example

Suppose we wish to find the probability of drawing either a king or a spade in a single draw from a pack of 52 playing cards.

We define the events A = 'draw a king' and B = 'draw a spade'

Since there are 4 kings in the pack and 13 spades, but 1 card is both a king and a spade, we have:

= 4/52 + 13/52 - 1/52 = 16/52

So, the probability of drawing either a king or a spade is 16/52 (= 4/13).

See also multiplication rule.

The multiplication rule is a result used to determine the probability that two events, A and B, both occur.

The multiplication rule follows from the definition of conditional probability.

The result is often written as follows, using set notation:

where:

P(A) = probability that event A occurs

P(B) = probability that event B occurs

= probability that event A and event B occur

P(A | B) = the conditional probability that event A occurs given that event B has occurred already

P(B | A) = the conditional probability that event B occurs given that event A has occurred already

For independent events, that is events which have no influence on one another, the rule simplifies to:

That is, the probability of the joint events A and B is equal to the product of the individual probabilities for the two events.

In many situations, once more information becomes available, we are able to revise our estimates for the probability of further outcomes or events happening. For example, suppose you go out for lunch at the same place and time every Friday and you are served lunch within 15 minutes with probability 0.9. However, given that you notice that the restaurant is exceptionally busy, the probability of being served lunch within 15 minutes may reduce to 0.7. This is the conditional probability of being served lunch within 15 minutes given that the restaurant is exceptionally busy.

The usual notation for "event A occurs given that event B has occurred" is "A | B" (A given B). The symbol | is a vertical line and does not imply division. P(A | B) denotes the probability that event A will occur given that event B has occurred already.

A rule that can be used to determine a conditional probability from unconditional probabilities is:

where:

P(A | B) = the (conditional) probability that event A will occur given that event B has occured already

= the (unconditional) probability that event A and event B both occur

P(B) = the (unconditional) probability that event B occurs

The result is often written as follows, using set notation:

where:

P(A) = probability that event A occurs

= probability that event A and event B both occur

= probability that event A and event B' both occur, i.e. A occurs and B does not.

Using the multiplication rule, this can be expressed as

P(A) = P(A | B).P(B) + P(A | B').P(B')

Bayes' Theorem is a result that allows new information to be used to update the conditional probability of an event.

Using the multiplication rule, gives Bayes' Theorem in its simplest form:

Using the Law of Total Probability:

P(A | B) =

P(B | A).P(A) P(B | A).P(A) + P(B | A').P(A')

where:

P(A) = probability that event A occurs

P(B) = probability that event B occurs

P(A') = probability that event A does not occur

P(A | B) = probability that event A occurs given that event B has occurred already

P(B | A) = probability that event B occurs given that event A has occurred already

P(B | A') = probability that event B occurs given that event A has not occurred already

---

Top of page | Main Contents page