Probability Distributions & Statistical Inference
Understanding how data behaves and what it tells us about the world
What is a Probability Distribution?
Think of it this way: if you could repeat an experiment millions of times, the probability distribution tells you the pattern that emerges — which outcomes are common, which are rare, and how the data spreads out.
Two Types of Random Variables
Takes on countable values (integers, whole numbers).
Examples: Number of heads in 10 coin flips, number of defective items in a batch, dice rolls.
Described by a Probability Mass Function (PMF):
$P(X = x)$ for each value $x$
Key property: $\displaystyle\sum_{\text{all } x} P(X = x) = 1$
Takes on any value in an interval (decimals, real numbers).
Examples: Height, weight, temperature, time.
Described by a Probability Density Function (PDF):
$f(x)$ where $P(a \le X \le b) = \int_a^b f(x)\,dx$
Key property: $\displaystyle\int_{-\infty}^{\infty} f(x)\,dx = 1$
Key Parameters: Mean, Variance, and Standard Deviation
Every probability distribution is characterized by parameters that describe its center and spread.
| Parameter | Discrete Formula | Continuous Formula | Meaning |
|---|---|---|---|
| Mean (Expected Value) $\mu = E(X)$ | $\displaystyle\sum x \cdot P(X=x)$ | $\displaystyle\int_{-\infty}^{\infty} x \cdot f(x)\,dx$ | The "center" or long-run average |
| Variance $\sigma^2 = \text{Var}(X)$ | $\displaystyle\sum (x - \mu)^2 \cdot P(X=x)$ | $\displaystyle\int_{-\infty}^{\infty} (x-\mu)^2 f(x)\,dx$ | Average squared distance from the mean |
| Standard Deviation $\sigma$ | $\sigma = \sqrt{\text{Var}(X)}$ | Spread in original units | |