Skip to content
Skip to page content
soylai.com

Probability — chapter notes

6 sections 4 min readBeta

1. Laws of probability and the addition theorem

For events of a sample space SS: 0≤P(A)≤10 \le P(A) \le 1, P(S)=1P(S) = 1, P(A′)=1−P(A)P(A') = 1 - P(A).

Addition theorem: P(A∪B)=P(A)+P(B)−P(A∩B)P(A \cup B) = P(A) + P(B) - P(A \cap B). For mutually exclusive events the overlap is empty and P(A∪B)=P(A)+P(B)P(A \cup B) = P(A) + P(B).

The overlap is subtracted once because it was counted twice.
LawStatement
Range0≤P(A)≤10 \le P(A) \le 1; P(S)=1P(S) = 1, P(∅)=0P(\varnothing) = 0
ComplementP(A′)=1−P(A)P(A') = 1 - P(A)
Addition theoremP(A∪B)=P(A)+P(B)−P(A∩B)P(A \cup B) = P(A) + P(B) - P(A \cap B)
Mutually exclusive eventsP(A∪B)=P(A)+P(B)P(A \cup B) = P(A) + P(B)

Formulas

  • Addition theorem
    P(A∪B)=P(A)+P(B)−P(A∩B)P(A \cup B) = P(A) + P(B) - P(A \cap B)

Key terms

Mutually exclusive events

Events that cannot happen together: their intersection is empty.

Complement

The event A′A' that AA does not happen.

2. Conditional probability

If P(B)>0P(B) > 0, the probability of AA given that BB has happened is P(A∣B)=P(A∩B)P(B)P(A \mid B) = \dfrac{P(A \cap B)}{P(B)}. With equally likely outcomes, P(A∣B)=n(A∩B)n(B)P(A \mid B) = \dfrac{n(A \cap B)}{n(B)}: restrict to the reduced sample space BB.

Properties: P(B∣B)=1P(B \mid B) = 1; P(A′∣B)=1−P(A∣B)P(A' \mid B) = 1 - P(A \mid B); the addition theorem holds given BB. In general P(A∣B)≠P(B∣A)P(A \mid B) \ne P(B \mid A).

In the imagined Dehradun survey, P(chess∣science)=38P(\text{chess} \mid \text{science}) = \dfrac{3}{8} but P(science∣chess)=916P(\text{science} \mid \text{chess}) = \dfrac{9}{16}.

Imagined survey data used throughout the chapter.
Chess clubNot in chess clubTotal
Science45457575120120
Commerce353545458080
Total8080120120200200
The reduced sample space has 11 outcomes, not 36.
Outcomes with at least one 4Sum 7?
(4,1),(4,2),(4,3),(4,4),(4,5),(4,6)(4, 1), (4, 2), (4, 3), (4, 4), (4, 5), (4, 6)(4,3)(4, 3)
(1,4),(2,4),(3,4),(5,4),(6,4)(1, 4), (2, 4), (3, 4), (5, 4), (6, 4)(3,4)(3, 4)
1111 outcomes in all22 favourable: 211\dfrac{2}{11}

Formulas

  • Definition
    P(A∣B)=P(A∩B)P(B),P(B)>0P(A \mid B) = \dfrac{P(A \cap B)}{P(B)}, \quad P(B) > 0

Key terms

Conditional probability

The probability of an event when another event is known to have happened.

Reduced sample space

The outcomes of the given event, which become the new sample space.

3. Multiplication theorem and independence

Multiplication theorem: P(A∩B)=P(A) P(B∣A)P(A \cap B) = P(A)\,P(B \mid A), and for three events P(A∩B∩C)=P(A) P(B∣A) P(C∣A∩B)P(A \cap B \cap C) = P(A)\,P(B \mid A)\,P(C \mid A \cap B). Use it for successive draws: without replacement the later factors change.

Independent events: P(A∩B)=P(A) P(B)P(A \cap B) = P(A)\,P(B), equivalently P(A∣B)=P(A)P(A \mid B) = P(A). If AA, BB are independent then so are AA and B′B', A′A' and BB, A′A' and B′B'; hence P(at least one)=1−P(A′) P(B′)P(\text{at least one}) = 1 - P(A')\,P(B').

Events with non-zero probabilities cannot be both mutually exclusive and independent.

Both Darjeeling: $\frac{3}{10} \cdot \frac{2}{9} = \frac{1}{15}$.
Equal values: independent. Unequal: dependent.
Event BP(B)P(B)P(A∩B)P(A \cap B)P(A) P(B)P(A)\,P(B)Independent?
Sum is 716\dfrac{1}{6}112\dfrac{1}{12}112\dfrac{1}{12}Yes
Sum is 8536\dfrac{5}{36}112\dfrac{1}{12}572\dfrac{5}{72}No

Formulas

  • Multiplication theorem
    P(A∩B)=P(A) P(B∣A)P(A \cap B) = P(A)\,P(B \mid A)
  • Independence
    P(A∩B)=P(A) P(B)P(A \cap B) = P(A)\,P(B)

Key terms

Independent events

Events for which the probability of both equals the product of their probabilities.

Dependent events

Events that are not independent: knowing one changes the chance of the other.

4. Theorem of total probability

If E1,…,EnE_1, \ldots, E_n form a partition of SS (pairwise disjoint, each P(Ei)>0P(E_i) > 0, union SS) and AA is any event, then

P(A)=∑i=1nP(Ei) P(A∣Ei)P(A) = \sum_{i=1}^{n} P(E_i)\,P(A \mid E_i).

In the imagined bolt factory: P(D)=0.5×0.02+0.3×0.03+0.2×0.05=0.029P(D) = 0.5 \times 0.02 + 0.3 \times 0.03 + 0.2 \times 0.05 = 0.029.

$P(D) = 0.01 + 0.009 + 0.01 = 0.029$.
$A$ is cut into the pieces $A \cap E_1$, $A \cap E_2$, $A \cap E_3$.

Formulas

  • Total probability
    P(A)=∑i=1nP(Ei) P(A∣Ei)P(A) = \sum_{i=1}^{n} P(E_i)\,P(A \mid E_i)

Key terms

Partition

Events that do not overlap, have non-zero probabilities and together make up the whole sample space.

Likelihood

P(A∣Ei)P(A \mid E_i): the probability of the evidence under a given cause.

5. Bayes' theorem

With the same partition and P(A)>0P(A) > 0:

P(Ei∣A)=P(Ei) P(A∣Ei)∑jP(Ej) P(A∣Ej)P(E_i \mid A) = \dfrac{P(E_i)\,P(A \mid E_i)}{\sum_j P(E_j)\,P(A \mid E_j)}.

P(Ei)P(E_i) is the prior, P(Ei∣A)P(E_i \mid A) the posterior. The posteriors add to 11. A posterior exceeds its prior exactly when P(A∣Ei)>P(A)P(A \mid E_i) > P(A).

For a rare condition even an accurate test gives a modest posterior: in the imagined cattle test, P(sick∣+)=1867P(\text{sick} \mid +) = \dfrac{18}{67}.

A Bayes table keeps every step visible.
MachinePriorLikelihoodProductPosterior
P0.50.50.020.020.0100.0101029\dfrac{10}{29}
Q0.30.30.030.030.0090.009929\dfrac{9}{29}
R0.20.20.050.050.0100.0101029\dfrac{10}{29}
Total110.0290.02911
$P(\text{sick} \mid +) = \frac{18}{67}$: most positives are healthy animals.
1000 animalsTest positiveTest negativeTotal
Sick1818222020
Healthy4949931931980980
Total676793393310001000

Formulas

  • Bayes' theorem
    P(Ei∣A)=P(Ei) P(A∣Ei)∑jP(Ej) P(A∣Ej)P(E_i \mid A) = \dfrac{P(E_i)\,P(A \mid E_i)}{\sum_j P(E_j)\,P(A \mid E_j)}

Key terms

Prior probability

The probability of a cause before the evidence is seen.

Posterior probability

The probability of a cause after the evidence is seen.

6. Random variable, distribution and mean

A random variable XX assigns a real number to each outcome. Its probability distribution lists the values xix_i with probabilities pip_i, where pi≥0p_i \ge 0 and ∑pi=1\sum p_i = 1.

The mean (expectation) is E(X)=∑xi piE(X) = \sum x_i\,p_i, the long-run average value of XX.

Examples: fused bulbs, E(X)=23E(X) = \dfrac{2}{3}; difference of two dice, E(X)=3518E(X) = \dfrac{35}{18}; P(X=x)=k(x2+1)P(X = x) = k(x^2 + 1) for x=0,1,2,3x = 0, 1, 2, 3 gives k=118k = \dfrac{1}{18} and E(X)=73E(X) = \dfrac{7}{3}.

The probabilities add to 1.
xxP(X=x)P(X = x)
0025\dfrac{2}{5}
11815\dfrac{8}{15}
22115\dfrac{1}{15}
  • P(X = x)
$E(X) = \frac{70}{36} = \frac{35}{18}$.

Formulas

  • Mean of a random variable
    E(X)=∑ixi pi,∑ipi=1E(X) = \sum_{i} x_i\,p_i, \quad \sum_i p_i = 1

Key terms

Random variable

A rule that gives a number to every outcome of an experiment.

Mean (expectation)

E(X)=∑xi piE(X) = \sum x_i\,p_i, the probability-weighted average of the values.

Exam tips

How ISC marks are awarded

  • Conditional probability and independence (2 marks): formula or reduced sample space, then the value; for independence show both sides of the product rule.
  • Total probability (2–3 marks): all causes with priors and likelihoods, then the sum.
  • Bayes (3–5 marks): total probability, the formula, the value, and an interpretation.
  • Distribution and mean (3–5 marks): each probability, the check that they add to 11, and the mean.

Common mistakes

  • Dividing by the wrong total, or confusing P(A∣B)P(A \mid B) with P(B∣A)P(B \mid A).
  • Treating draws without replacement as independent.
  • Confusing mutually exclusive with independent.
  • Reporting the prior instead of the posterior.
  • Taking a simple average of the values instead of the weighted mean.