Stanford Introduction to Statistics Notes

Introduction to Statistics — Exam Study Notes

Course: Stanford's Introduction to Statistics (Guenther Walther, Stanford Online) Style: one complete set of handwritten-style study notes, organized the way you actually learn the subject — from describing data to testing hypotheses. Every formula comes with its conditions, every method with steps, every topic with worked examples (all numbers machine-verified) and exam traps.

Notation: μ population mean, σ population sd, x̄ sample mean, s sample sd, n sample size, N population size, p population proportion, p̂ sample proportion, df degrees of freedom.

---

PART 1 — DESCRIBING DATA

1.1 Why statistics

Statistics = collecting, organizing, analyzing, interpreting data to make decisions under uncertainty. Two branches:

• Descriptive — summarize what the data show (this part).
• Inferential — use a sample to say something about the population (later parts). Vocabulary you must not mix up:
• Population = the entire group of interest. Parameter = a number describing it (μ, σ, p). Fixed, usually unknown.
• Sample = the part we actually measure. Statistic = a number computed from it (x̄, s, p̂). It changes from sample to sample.
• Exam tip: "the average height of ALL students" → parameter; "average of the 50 we measured" → statistic.

1.2 Graphs and when to use each

• Distribution shapes: symmetric, right-skewed (long right tail), left-skewed, unimodal / bimodal.
• Trap: a histogram with two peaks (bimodal) often means two mixed populations — consider splitting.

1.3 Measures of center

• Mean: x̄ = Σxᵢ/n. Uses every value; pulled toward the tail.
• Median: middle of sorted data (average the two middle values if n even). Robust to outliers.
• Mode: most frequent; the only average for categorical data.
• Relationship (memorize the picture): right-skewed → mean > median; left-skewed → mean < median; symmetric → mean ≈ median.
• When to use: skewed data or outliers → report median (+IQR); symmetric → mean (+sd).

Example 1.1 (center under skew). Salaries: 30, 35, 38, 40, 250 (in $k). Mean = 78.6, median = 38. The single outlier drags the mean above every real value except the outlier itself → report the median.

1.4 Measures of spread

• Range = max − min. Uses only 2 points — crude.
• IQR = Q3 − Q1 = range of the middle 50%. Robust to outliers.
• Five-number summary: min, Q1, median, Q3, max (what a boxplot shows).
• Variance & standard deviation — the core. Deviation of a point = xᵢ − mean. Deviations always sum to 0, so we square them:

• sd = √variance → back to original units (variance has squared units).
• Why n−1 ("degrees of freedom"): using x̄ instead of the true μ loses one free piece of information; dividing by n−1 makes s² unbiased for σ². If an exam says "a sample of…" you MUST use n−1.
• sd is sensitive to outliers (squared deviations); alternatives that resist them: MAD, IQR.
• sd = 0 ⟺ every value identical.
• Adding the same constant c to all data: mean +c, sd unchanged. Multiplying all by c: mean ×c, sd ×|c|.

Example 1.2 (full sd calculation — the canonical one). Data: 2, 4, 4, 4, 5, 5, 7, 9 (n = 8).

1. mean = 40/8 = 5
2. deviations: −3, −1, −1, −1, 0, 0, 2, 4 → sum = 0 ✓
3. squared: 9, 1, 1, 1, 0, 0, 4, 16 → SS = 32
4. treat as population: σ² = 32/8 = 4 → σ = 2
5. treat as sample: s² = 32/7 ≈ 4.5714 → s ≈ 2.13809 Both answers are correct given the wording — state the denominator you used.

Example 1.3 (five-number summary + outlier fences). Sorted data (n = 10): 7, 15, 36, 39, 40, 41, 42, 43, 47, 49.

• min 7, max 49; median = (40+41)/2 = 40.5
• Q1 = 30.75, Q3 = 44 (quartile positions 2.75 and 8.25 in the sorted list, interpolated) → IQR = 13.25
• Fences: Q1 − 1.5·IQR = 10.875; Q3 + 1.5·IQR = 63.875
• 7 < 10.875 → 7 is an outlier; no high outliers.
• mean = 35.9 < median = 40.5 → slight left skew (the low outlier pulls the mean).

1.5 Position: percentiles and standardized scores

• p-th percentile: p% of values fall below it. Q1 = P25, median = P50, Q3 = P75.
• z-score: z = (x − x̄)/s (sample) or (x − μ)/σ (population) — "how many sds from the mean". Positive = above, negative = below, 0 = at the mean. See Part 4.

1.6 Two famous "rules"

• Chebyshev's inequality — works for ANY distribution, any shape:
  • at least 75% of data within μ ± 2σ
  • at least 89% within μ ± 3σ
  • at least 95% within μ ± 4.5σ (general: at least 1 − 1/k² within k sd)
• Empirical rule (68–95–99.7) — ONLY for bell-shaped symmetric distributions:
  • ≈68% within 1σ, ≈95% within 2σ, ≈99.7% within 3σ (exact normal values: 68.27%, 95.45%, 99.73%)
• Exam trap: "At least 75%…" → Chebyshev (works always). "About 95%…" → empirical rule (needs normality). If the data are skewed and the question says "at least", the answer uses Chebyshev.

1.7 Part 1 checklist

• Can compute mean, median, mode, range, IQR, five-number summary by hand.
• Can compute σ and s from a small dataset and say which one you used.
• Can read a boxplot/histogram: skew, outliers, bimodality.
• Know which of Chebyshev vs empirical rule a question is asking for.
• Know how adding/multiplying constants changes mean and sd.

---

PART 2 — COLLECTING DATA: SAMPLING AND EXPERIMENTS

2.1 Why design matters

Bad data + fancy analysis = confident nonsense. The design fixes the ceiling of what you can conclude.

2.2 Sampling designs

• Simple random sample (SRS): every set of n individuals equally likely to be chosen (label + random number table / RNG).
• Stratified random sample: split population into strata that differ meaningfully (e.g., grade levels), SRS inside each stratum → guarantees representation, more precise.
• Cluster sample: randomly pick whole clusters (e.g., classrooms), measure everyone inside → cheaper; less precise per n.
• Systematic sample: every k-th individual after random start.
• Convenience / voluntary response / self-selected → biased samples, do NOT represent the population.

2.3 Bias catalog (name the bias in one phrase)

• Sampling error (chance error): sample-to-sample fluctuation — inevitable, shrinks with n.
• Nonsampling error: bad design, biased questions, nonresponse — does NOT shrink with bigger n.

2.4 Observational study vs randomized experiment

• Observational: measure, don't intervene. Can find association; confounding variables can hide behind the observed link → no clean causation.
• Randomized controlled experiment (RCT):
  1. random assignment to treatment/control (balances confounders on average)
  2. control group (baseline; placebo when relevant)
  3. blinding (subjects don't know) and double-blinding (+ experimenters don't know)
  4. replication (enough subjects)
• Placebo effect: improvement from expectation, not treatment. Placebo + blinding controls it.
• Block design: group similar subjects first, randomize within blocks (like stratification for experiments).
• Lurking/confounding variable: affects both the explanatory and response variables, faking or masking a relation (classic: ice cream sales ↔ drownings, confounder = summer heat).

2.5 Exam decision rule

• "Can we conclude causation?" → ONLY if randomized experiment. Otherwise say "association; possible confounding."
• "Name the bias" → pick from 2.3 table and say why.
• "Two sampling designs compared" → stratified = precision by known groups; cluster = convenience by location.

---

PART 3 — PROBABILITY

3.1 Interpretation and axioms

• Frequentist: long-run relative frequency. (Also subjective/Bayesian for single events.)
• Axioms: 0 ≤ P(A) ≤ 1; P(whole sample space S) = 1; disjoint events add.
• Equally likely outcomes: P(A) = (# outcomes in A)/(# outcomes in S). The "equally likely" assumption is an assumption — check it (a loaded die violates it).

3.2 Complement and addition

• Complement: P(Aᶜ) = 1 − P(A). "At least one" problems are complements in disguise (below).
• General addition: P(A ∪ B) = P(A) + P(B) − P(A ∩ B) (subtract the overlap counted twice).
• Mutually exclusive (disjoint): P(A ∩ B) = 0, so P(A ∪ B) = P(A) + P(B).

Example 3.1. Card deck: P(King or Heart) = 4/52 + 13/52 − 1/52 = 16/52 ≈ 0.3077. (The King of Hearts is in both sets.)

3.3 Conditional probability and multiplication

• Conditional: P(A|B) = P(A ∩ B)/P(B) — restrict the world to B, then re-measure A.
• General multiplication: P(A ∩ B) = P(A) · P(B|A) = P(B) · P(A|B).
• Independent events: P(A|B) = P(A) (B carries no information), equivalently P(A ∩ B) = P(A)P(B).
• Trap: independent ≠ mutually exclusive. Disjoint positive-probability events are DEPENDENT (knowing one happened rules the other out).
• Sampling note: draws with replacement → independent; without replacement → dependent (hypergeometric flavor).

Example 3.2. P(A) = 0.5, P(B) = 0.4, independent: P(A ∩ B) = 0.2, P(A ∪ B) = 0.5 + 0.4 − 0.2 = 0.7, P(A|B) = 0.5.

3.4 Contingency tables (conditional probability home turf)

Example 3.3. n = 100 students. Rows gender, columns exam result:

• P(pass) = 65/100 = 0.65 (marginal)
• P(pass AND woman) = 35/100 = 0.35 (joint)
• P(pass | woman) = 35/50 = 0.70 (conditional: restrict to the woman row)
• Compare conditional rates to judge association: 0.70 vs 30/50 = 0.60 → women pass more often in this sample.

3.5 "At least one" — complement trick

• P(at least one success in n tries) = 1 − P(no successes) = 1 − (1−p)ⁿ. Example 3.4. Free throws, hit rate 70%, shoot 4: P(at least one) = 1 − 0.3⁴ = 1 − 0.0081 = 0.9919.

3.6 Total enumeration & counting

• When the sample space is small, LIST it and count.
• Multiplication rule of counting: stages multiply (outfits = 3 shirts × 2 pants = 6).
• Permutations (order matters): P(n,k) = n!/(n−k)!. Example: 5P3 = 5·4·3 = 60.
• Combinations (order doesn't matter): C(n,k) = n!/(k!(n−k)!). Example: 5C3 = 10.
• Ask: "would rearranging the chosen items create a different outcome?" Yes → permutation; No → combination. (Part 4's binomial coefficient is a combination.)

3.7 Bayes' rule

P(A|B) = P(B|A)·P(A) / P(B) — invert the conditioning: from P(cause→evidence) to P(evidence→cause).

Worked Bayes example (verified). Disease prevalence 1% (P(D)=0.01). Test: sensitivity 99% (P(+|D)=0.99), specificity 95% (P(−|healthy)=0.95, so P(+|healthy)=0.05).

• P(D ∩ +) = 0.01 × 0.99 = 0.0099
• P(healthy ∩ +) = 0.99 × 0.05 = 0.0495
• P(+ ) = 0.0099 + 0.0495 = 0.0594
• P(D | +) = 0.0099/0.0594 ≈ 0.1667 = 16.7% Interpretation for exams: with rare conditions, most positives are false positives even for accurate tests — always multiply by the base rate. False-negative trap: P(−|D) = 0.01 is NOT the same as P(D|−).

3.8 Part 3 checklist

• addition with and without overlap; complement for "at least one"
• joint vs marginal vs conditional on a contingency table
• independent vs disjoint (they are different!)
• permutation vs combination
• Bayes with base rates

---

PART 4 — RANDOM VARIABLES AND DISTRIBUTIONS: BINOMIAL AND NORMAL

4.1 Random variables, quickly

• Random variable (RV): a number assigned to each outcome. Discrete (countable: 0,1,2,…) vs continuous (any value in an interval).
• Discrete: probability mass function p(x) with Σp(x)=1. Continuous: density f(x) with area=1; P(X = exact value) = 0, only intervals have probability.
• Expected value (long-run average): E(X) = Σ x·p(x) (discrete). Rule: E(aX+b) = aE(X)+b.
• Variance: Var(X) = E(X²) − [E(X)]². sd = √Var. Rule: Var(aX+b) = a²Var(X).

4.2 The normal curve

• X ~ N(μ, σ): continuous, bell-shaped, symmetric about μ (so mean = median = mode), total area = 1.
• Parameters: μ shifts the curve left/right; σ stretches (bigger σ = flatter, wider).
• There are infinitely many normals — they all relate to ONE standard curve.
• Empirical rule (exact values): within ±1σ: 68.27%, ±2σ: 95.45%, ±3σ: 99.73%.

4.3 Standardizing (the z-score) — the key move

• z = (x − μ)/σ measures the distance from the mean in units of sd.
• Backward: x = μ + zσ.
• Z ~ N(0, 1) is the standard normal; every normal question becomes a Z question in two steps:
  1. convert x → z
  2. read area from the z-table (or software). Tables give P(Z ≤ z) by default.
• Going backwards (inverse problems): "the top 2.5% cut-off" → find z with area 0.95 to the left → z = 1.96 → x = μ + 1.96σ. Memorize: 90% → 1.645, 95% → 1.96, 99% → 2.576 (two-sided critical values), and 80/90/95/99th percentiles → 0.84 / 1.28 / 1.645 / 2.33.

Example 4.1 (forward). Heights X ~ N(64, 3). x = 70.4: z = (70.4 − 64)/3 ≈ 2.13 → P(X > 70.4) = 1 − P(Z ≤ 2.13) ≈ 1 − 0.9834 = 0.0166 (about 1.7% — unusually tall).

Example 4.2 (backward). z = −1.5 → x = 64 + (−1.5)(3) = 59.5.

Example 4.3 (interval). P(58 < X < 70) for N(64,3): z = −2 and +2 → 0.9545 (empirical rule's "95%").

4.4 The binomial setting — run the checklist

1. Fixed number of trials n.
2. Each trial → success/failure.
3. Same success probability p every trial.
4. Trials independent. All four → Binomial(n, p). If sampling without replacement from a small population, independence fails (that's hypergeometric territory — for big populations the effect is negligible).

4.5 Binomial formulas

• Binomial coefficient C(n,k) = n!/(k!(n−k)!) — the number of orderings that give k successes.
• PMF: P(X = k) = C(n,k) pᵏ (1−p)ⁿ⁻ᵏ
• Mean = np. Variance = np(1−p). sd = √(np(1−p)).
• "At least one" → 1 − (1−p)ⁿ.

Example 4.4 (exact). n = 10 coin flips (p = 0.5): P(exactly 4 heads) = C(10,4)(0.5)¹⁰ = 210/1024 ≈ 0.2051. P(at most 1 head) = (1+10)/1024 ≈ 0.0107.

Example 4.5 (mean/sd). n = 100 free throws, p = 0.2: mean = 20, sd = √(100·0.2·0.8) = 4. (You expect 20 ± about 4.)

4.6 Normal approximation to the binomial

• Why: factorials explode; a normal curve with the same mean/sd is close when n is large.
• Condition (quote it in exams): np ≥ 10 AND n(1−p) ≥ 10.
• Method: X ≈ N(np, √(np(1−p))); then standardize. Add/subtract 0.5 (continuity correction) for exact-step accuracy when using tables. Example 4.6. n = 100, p = 0.2, P(X ≤ 15): mean 20, sd 4; with continuity correction z = (15.5 − 20)/4 = −1.125 → area ≈ 0.130.

4.7 Concept boundary (exam favorite)

• "Standardizing a dataset" (Part 1 / Part 4): a descriptive transform of DATA → z-scores.
• "Normal approximation to binomial" (Part 4): approximating a COUNT distribution with a normal curve.
• Same curve, totally different questions. Don't merge them in an answer.

---

PART 5 — FROM SAMPLE TO POPULATION: SAMPLING DISTRIBUTIONS AND THE CLT

5.1 Parameter vs statistic, once more

Parameter (μ, σ, p) — fixed but unknown. Statistic (x̄, s, p̂) — random, changes sample to sample. The uncertainty of inference lives in the statistic.

5.2 The three histograms (classic exam picture)

1. the population distribution (any shape),
2. the data of one sample (n points, shape mirrors population),
3. the sampling distribution of x̄ (from many samples) — this one goes normal as n grows (CLT). Know which one a question describes. Confusing #2 with #3 is the classic error.

5.3 Expected value and standard error

• SE = the sd of the statistic across repeated samples (called standard error because σ/p are usually estimated).
• Square root law: to cut SE in half, quadruple n. Ten times the data → about 3.2× precision.

5.4 Law of Large Numbers vs Central Limit Theorem

• LLN: x̄ converges to μ as n → ∞ (accuracy).
• CLT: the SHAPE of x̄'s distribution becomes N(μ, σ/√n) as n grows — regardless of the population's shape (why the normal curve is everywhere). Rule of thumb n ≥ 30 for moderate skew; symmetric populations converge faster; heavy-skew needs more.
• What CLT does NOT say: the population becomes normal; or small-n data are normal.

5.5 Using it

Example 5.1. Bottles filled with μ = 500 ml, σ = 10. n = 25 bottles → SE = 10/5 = 2. P(x̄ < 498): z = (498−500)/2 = −1 → area 0.1587. (Note: a single bottle at 498 would use σ=10 → z=−0.2 → 0.4207. n in the denominator is the whole point.)

Example 5.2 (proportion). p = 0.5, n = 100: SE = √(0.25/100) = 0.05. P(p̂ ≥ 0.58): z = (0.58−0.5)/0.05 = 1.6 → 1 − 0.9452 = 0.0548.

---

PART 6 — RELATIONSHIPS: CORRELATION AND REGRESSION

6.1 Correlation r

• r measures linear association between two quantitative variables: −1 ≤ r ≤ 1. Sign = direction; magnitude = tightness of the cloud around a line.
• r is unit-free, symmetric in x/y, and NOT resistant to outliers.
• r = 0 means "no LINEAR relation" — data can sit on a perfect parabola and give r ≈ 0.
• Correlation is NOT causation (lurking variables; reverse direction; coincidence).

6.2 Least-squares regression

• Line: ŷ = a + bx, chosen to minimize Σ(y − ŷ)² (squared vertical residuals).
• Slope: b = r · (s_y/s_x) — "rise in y per unit x, scaled by the sd ratio".
• Intercept: a = ȳ − b·x̄. The line always passes through (x̄, ȳ).
• Interpretation: b = average change in y per one-unit increase in x (within the observed x-range — extrapolation is dangerous).
• Predicting y from x and predicting x from y give DIFFERENT lines (unless |r| = 1).

Worked example (verified). (x, y): (1,2), (2,4), (3,5), (4,4), (5,5).

• x̄ = 3, ȳ = 4; Sxy = 6, Sxx = 10, Syy = 6
• b = 6/10 = 0.6; a = 4 − 0.6·3 = 2.2 → ŷ = 2.2 + 0.6x
• r = 6/√(10·6) = 6/√60 ≈ 0.7746; r² ≈ 0.60
• Prediction at x = 6: ŷ = 2.2 + 3.6 = 5.8. Residual at (1,2): 2 − 2.8 = −0.8.

6.3 Reading r²

• r² = fraction of y's variance explained by the linear model. r = 0.77 → r² = 0.60 → "60% of the variation in y is explained by x (linearly)".

6.4 Diagnostics (what the course tests)

• Residual plot (residual vs x): should look like noise.
  • funnel shape → heteroscedasticity → try transforming y (log).
  • curve → wrong functional form → try quadratic, etc.
• Outlier: far in y (big residual). Influential point: far in x AND pulls the line (remove it and see).
• Regression to the mean / regression fallacy: extreme performances are followed by more ordinary ones for statistical reasons, not because of the "reward" or "punishment" in between. Classic exam story (pilots praised after bad landings improve — they would have anyway).

---

PART 7 — ESTIMATION: CONFIDENCE INTERVALS AND THE BOOTSTRAP

7.1 What a CI is (interpretation points!)

• A confidence interval (CI) is estimate ± margin of error, built so that the PROCEDURE captures the true parameter in C% of repeated samples.
• Correct phrasing (95% CI): "We are 95% confident that the interval from … to … contains the true mean." — "95% confident" describes the method's long-run success rate.
• WRONG: "there is a 95% probability that μ is in this interval" (μ is fixed, not random).
• WRONG: "95% of the data lie in this interval" (that's a data range, not a CI for a mean).

7.2 The master template

estimate ± (critical value) × (standard error) Every CI in this course is this template with different estimate/SE.

• Mean, σ known or large n: x̄ ± z* · σ/√n
• Mean, σ unknown: x̄ ± t* · s/√n, df = n−1 (t has fatter tails; t* > z*)
• Proportion: p̂ ± z* · √(p̂(1−p̂)/n) (use p̂ in the SE; conditions np̂ ≥ 10, n(1−p̂) ≥ 10)
• Critical values: C = 90% → z* = 1.645; 95% → 1.960; 99% → 2.576.

Example 7.1 (mean). n = 100, x̄ = 50, s = 10: CI = 50 ± 1.96·(10/√100) = 50 ± 1.96 = (48.04, 51.96).

Example 7.2 (proportion). p̂ = 0.60, n = 100: SE = √(0.6·0.4/100) = 0.049; CI = 0.6 ± 1.96·0.049 = (0.504, 0.696).

7.3 What moves the width

• Confidence ↑ → interval wider (more z*). Sample size ↑ → narrower (SE ∝ 1/√n). More variability (s) → wider.
• Sample size planning: n = (z*·σ/E)², E = desired margin of error. Always round UP. Example 7.3. Want E = 3 with σ = 15 at 95%: n = (1.96·15/3)² = 96.04 → n = 97.

7.4 The bootstrap (modern course content)

• When no formula SE exists: resample the observed data WITH replacement, same n, many times (e.g., 10,000); each resample gives a statistic; the spread of those = SE estimate.
• Percentile CI: take the 2.5th and 97.5th percentiles of the bootstrap statistics (for 95%).
• Logic ("plug-in principle"): the sample is the best proxy for the population we have.
• Limits: bootstrap can't rescue a biased or tiny non-representative sample.

---

PART 8 — TESTING HYPOTHESES

8.1 The logic

1. H₀ (null): no effect / no difference / status quo (the skeptical default).
2. Hₐ (alternative): what we suspect; one-sided (>, <) or two-sided (≠) — decide BEFORE seeing data.
3. Compute a test statistic measuring how far the estimate is from the H₀ value, in SE units:
   • mean (σ known): z = (x̄ − μ₀)/(σ/√n)
   • mean (σ unknown): t = (x̄ − μ₀)/(s/√n), df = n − 1 ← the default in practice
   • proportion: z = (p̂ − p₀)/√(p₀(1−p₀)/n) (uses p₀, the HYPOTHESIZED p, not p̂)
4. p-value = probability, computed assuming H₀, of a result this extreme or more (one-sided: one tail; two-sided: double the tail you got, or both tails).
5. Decision: p ≤ α → reject H₀ (statistically significant); p > α → fail to reject.

8.2 The five interpretation laws (most-tested sentences)

1. The p-value is P(data this extreme | H₀ true) — NOT P(H₀ | data).
2. "Fail to reject H₀" does NOT prove H₀ true — your test just lacked evidence (absence of evidence ≠ evidence of absence).
3. Statistical significance ≠ practical importance (big n makes tiny differences significant).
4. α = P(Type I error) = rejecting a true H₀. β = P(Type II) = missing a real effect. Power = 1 − β = detecting a real effect; power grows with n and with the true effect size.
5. Results are about parameters (μ, p), not about the sample itself.

Example 8.1 (proportion z-test, verified). Claim: coin fair (p₀ = 0.5). n = 100 flips, 56 heads. p̂ = 0.56.

• SE = √(0.5·0.5/100) = 0.05 → z = (0.56 − 0.5)/0.05 = 1.2
• two-sided p = 2·(1 − Φ(1.2)) ≈ 0.230 → fail to reject at α = 0.05. (56/100 heads is unremarkable.)

Example 8.2 (t-test, verified). H₀: μ = 100; n = 25, x̄ = 103, s = 10.

• t = (103 − 100)/(10/√25) = 1.5, df = 24 → two-sided p ≈ 0.147 → fail to reject at 0.05. Steps to write: hypotheses → df → statistic → p → decision → sentence in context ("we do not have evidence that the mean differs from 100").

8.3 The other two designs

• Two independent samples (two-sample z/t): statistic = difference of estimates; SE adds variances. Example 8.3 (two proportions, verified). Site A: 70/100 conversions; Site B: 55/100. Pooled p̂ = 125/200 = 0.625; SE = √(0.625·0.375·(1/100+1/100)) ≈ 0.0685; z = (0.70 − 0.55)/0.0685 ≈ 2.19 → one-sided p ≈ 0.014 → significant at 0.05.
• Matched pairs: pair up subjects (before/after, twins) → take DIFFERENCES → run a ONE-sample t on the differences. Don't use two-sample formulas on paired data (loses power).

---

PART 9 — RESAMPLING (MONTE CARLO AND BOOTSTRAP)

(A computer-first alternative to the formula methods of Parts 7–8.)

9.1 Monte Carlo simulation

• To estimate P(A) or E(X): simulate the random process many times; the simulated frequency/average converges to the truth (Law of Large Numbers).
• Error shrinks like 1/√(#simulations) — the square root law again.

9.2 The bootstrap (deep dive)

Algorithm (percentile bootstrap for a CI):

1. Draw a resample of size n from the data WITH replacement.
2. Compute the statistic (mean, median, correlation…).
3. Repeat B times (B = 10,000 typical).
4. SE estimate = sd of the B statistics; CI = 2.5/97.5 percentiles of them.

• Why it works: the empirical distribution (the sample) is the best available stand-in for the population (plug-in principle).
• What it buys: CIs for ANY statistic — medians, ratios, correlations — where closed formulas don't exist.
• What it can't do: fix bias, tiny samples, or dependence in the data.
• Course contrast: bootstrap (no distribution assumption, computer) vs formulas (normal-based, closed form). Same logic as CI: statistic ± 2·SE.

---

PART 10 — CATEGORICAL DATA: CHI-SQUARE TESTS

10.1 Setup

Categorical data come as COUNTS in categories. The three tests all compare observed counts O to expected counts E: χ² = Σ (O − E)²/E (large χ² = observed far from expected = evidence against H₀)

10.2 The three tests (same formula, different H₀)

• Expected count in a two-way table: E = (row total × column total)/grand total.
• Conditions: all expected counts ≥ 5 (approx.); data are counts (not percentages).
• The χ² distribution is right-skewed; big statistic → small p → reject H₀.

Example 10.1 (goodness of fit, verified). Is a die fair? 60 rolls: faces (1..6) observed 8, 9, 10, 11, 12, 10. H₀: p = 1/6 each → E = 10 per face. χ² = (4+1+0+1+4+0)/10 = 10/10 = 1.0, df = 5. Critical value at α = 0.05 is 11.07 → 1.0 ≪ 11.07 → the die looks fair.

Example 10.2 (independence 2×2, verified).

E(pass,men) = 50·65/100 = 32.5; E(fail,men) = 17.5; E(pass,women) = 32.5; E(fail,women) = 17.5. χ² = (2.5²/32.5)·2 + (2.5²/17.5)·2 = 0.3846 + 0.7143 ≈ 1.10, df = 1. Critical value 3.84 → NOT significant at 0.05: no evidence of a gender–pass association in this data.

10.3 Traps

• χ² significance does NOT tell you direction or which cells differ (follow-up with per-cell comparisons).
• Counts, not proportions, go into χ².
• df for GOF is categories−1, NOT (r−1)(c−1) — a classic mix-up.

---

PART 11 — COMPARING SEVERAL MEANS: ONE-WAY ANOVA

11.1 The question

Compare k ≥ 2 group means with ONE test: H₀: μ₁ = μ₂ = … = μ_k vs Hₐ: at least one differs.

11.2 The idea (say it in words)

Compare between-group variation to within-group variation. If group means truly differ, between-group variation is large relative to the noise inside groups.

F = MSB / MSW = [SSB/(k−1)] / [SSW/(N−k)]

• SSB = Σ n_g (mean_g − grand mean)² (between)
• SSW = Σ Σ (x − mean_g)² (within, pooled over groups)
• df: (k−1, N−k). Large F → reject H₀.

Worked example (verified). Groups: A = {3,5,7}, B = {5,7,9}, C = {8,9,10}.

• means 5, 7, 9; grand mean = 7
• SSB = 3(5−7)² + 3(7−7)² + 3(9−7)² = 12+0+12 = 24
• SSW = (4+0+4) + (4+0+4) + (1+0+1) = 18
• F = (24/2)/(18/6) = 12/3 = 4.0, df (2, 6). Critical value at 0.05 is 5.14 → 4.0 < 5.14 → fail to reject (borderline).

11.3 Conditions and follow-up

• Assumptions: independent samples; each population ≈ normal; equal variances.
• Rejecting H₀ does NOT say which pairs differ → multiple comparisons (Part 12) locate them.
• Two groups: F = t² (ANOVA generalizes the two-sample t-test).

---

PART 12 — MULTIPLE COMPARISONS AND REPRODUCIBILITY

12.1 The problem

Each test at α = 0.05 has a 5% false-positive rate. Run m independent tests: P(at least one false positive) = 1 − 0.95ᵐ → 40 tests ≈ 87%. With big data you WILL find "significant" results by luck (data snooping). This is the statistical heart of the reproducibility crisis.

12.2 The fixes

• Bonferroni: test each at α/m. Dead simple, very conservative (misses real effects). Example: 20 tests at overall 0.05 → each at 0.0025.
• False Discovery Rate (Benjamini–Hochberg): control the expected PROPORTION of false discoveries among your rejections (q = 0.05 typical). Procedure: sort p-values p(1)…p(m); find the largest i with p(i) ≤ (i/m)·q; reject the first i.
• Data splitting (holdout): explore on half the data, confirm hypotheses on the untouched half.
• Pre-registration: commit to hypotheses/analysis before seeing data (cultural fix).

12.3 Exam phrasing

• "Why not many t-tests?" → inflation of Type I error; give Bonferroni as the fix.
• "Bonferroni vs FDR?" → Bonferroni controls ANY false positive (family-wise); FDR controls the RATE of false positives among findings (more power for big m).

---

APPENDIX A — ONE-PAGE FORMULA SHEET

DESCRIPTIVE
  mean x̄ = Σx/n            median = middle (sorted)     mode = most frequent
  range = max − min         IQR = Q3 − Q1
  σ² = Σ(x−μ)²/N            s² = Σ(x−x̄)²/(n−1)          sd = √variance
  outlier fences: Q1 − 1.5·IQR,  Q3 + 1.5·IQR

POSITION
  z = (x − μ)/σ             x = μ + zσ
  Chebyshev: ≥ 1 − 1/k² within k sd (any data)
  Empirical rule (normal only): 68% / 95% / 99.7% within 1/2/3σ

PROBABILITY
  P(Aᶜ) = 1 − P(A)
  P(A∪B) = P(A) + P(B) − P(A∩B)         (exclusive: subtract 0)
  P(A∩B) = P(A)·P(B|A)                   (independent: = P(A)P(B))
  P(at least one) = 1 − (1−p)ⁿ
  Bayes: P(A|B) = P(B|A)·P(A)/P(B)
  counting: nPk = n!/(n−k)!   nCk = n!/(k!(n−k)!)

RANDOM VARIABLES
  E(X) = Σ x·p(x)        Var(X) = E(X²) − (E(X))²
  E(aX+b) = aE(X)+b      Var(aX+b) = a²Var(X)

BINOMIAL (check 4 conditions first)
  P(X=k) = C(n,k)pᵏ(1−p)ⁿ⁻ᵏ     mean np     sd √(np(1−p))
  normal approx if np ≥ 10 and n(1−p) ≥ 10

SAMPLING DISTRIBUTIONS / CLT
  SE(x̄) = σ/√n         SE(sum) = σ√n         SE(p̂) = √(p(1−p)/n)
  x̄ ≈ N(μ, σ/√n) for large n regardless of population shape

CONFIDENCE INTERVALS    estimate ± z*·SE
  mean: x̄ ± z*·σ/√n  (σ unknown → t*, df n−1)
  proportion: p̂ ± z*·√(p̂(1−p̂)/n)
  z*: 1.645 (90%), 1.96 (95%), 2.576 (99%)
  sample size: n = (z*σ/E)², round up

TESTS
  z = (estimate − hypothesized)/SE       t = (x̄−μ₀)/(s/√n), df = n−1
  p-value = P(data this extreme | H₀)     decision: p vs α
  Type I: reject true H₀ (α)   Type II: keep false H₀ (β)   Power = 1−β
  two proportions: pooled p̂ for SE; matched pairs → one-sample on differences

CHI-SQUARE   χ² = Σ(O−E)²/E     E = row×col/total
  GOF df = categories−1;  homogeneity/independence df = (r−1)(c−1)
  condition: all E ≥ 5

ANOVA   F = MSB/MSW = [SSB/(k−1)]/[SSW/(N−k)]
  SSB = Σ n_g(mean_g − grand)²     SSW = pooled within-group squares

REGRESSION
  b = r·(s_y/s_x)     a = ȳ − b·x̄     ŷ = a + bx     passes through (x̄, ȳ)
  r² = explained variance fraction     residual = y − ŷ

MULTIPLE COMPARISONS
  family-wise: Bonferroni α/m        discovery rate: FDR (Benjamini–Hochberg)
  inflation: P(≥1 false) = 1 − (1−α)ᵐ

APPENDIX B — WORKED MINI-EXAM (test yourself, all verified)

1. Data 2,4,4,4,5,5,7,9 — compute s. → SS=32, s=√(32/7)≈2.138
2. Same data are the whole population — σ? → √(32/8) = 2
3. Fair-looking test: prevalence 2%, sensitivity 90%, false-positive rate 5%. P(disease|+)? → (0.018)/(0.018+0.049) = 0.018/0.067 ≈ 0.269
4. n=50, p̂=0.44 — is 0.5 plausible at α=0.05? → SE=√(0.25/50)=0.0707, z=(0.44−0.5)/0.0707≈−0.85, p≈0.40 → plausible
5. 95% CI for mean, n=64, x̄=120, σ=16 → 120 ± 1.96·2 = (116.08, 123.92)
6. How large n for ME = 1 with σ = 16 at 95%? → (1.96·16)² ≈ 983.4 → n = 984
7. Binomial: n=20, p=0.3, P(X=5)? → C(20,5)·0.3⁵·0.7¹⁵ = 15504·0.00243·0.00475 ≈ 0.179
8. χ² GOF: 100 rolls, all faces observed equally → χ² ≈ 0 by construction; df = 5
9. Which design allows causal claims? → randomized controlled experiment
10. Mean=40.5, sd=7 — at least what % within (26.5, 54.5)? → that's ±2σ → Chebyshev: ≥75%

APPENDIX C — TRAP CHECKLIST (read the night before)

1. "sample" → n−1 denominator. "population/entire class" → N. Read the wording.
2. Empirical rule needs near-normal symmetric data; "at least" questions → Chebyshev.
3. Independent ≠ mutually exclusive (disjoint events are dependent).
4. Conditional probability restricts the denominator — P(A|B) ≠ P(A)P(B) unless independent.
5. In binomial SE and proportion tests: p(1−p), never p·p. In hypothesis tests for p, use p₀ from H₀ inside the SE.
6. CLT is about the sampling distribution of x̄ — the population does not become normal.
7. SE has √n in the denominator: halving the error quadruples n (square root law).
8. 95% confidence describes the METHOD's long-run capture rate, not a probability statement about this interval, and it is NOT a data range.
9. p-value: P(data|H₀). Never P(H₀|data). Fail-to-reject ≠ accept. Significant ≠ important.
10. Two-sided p = double the one-tail area. Choose sided-ness BEFORE data.
11. Matched pairs → one-sample t on differences; two independent samples → two-sample t. Don't swap them.
12. χ² uses COUNTS with all E ≥ 5; GOF df = k−1, table tests df = (r−1)(c−1).
13. ANOVA "significant" means SOME means differ — follow with multiple comparisons.
14. Bonferroni divides α by m; running more tests inflates false positives (1−0.95ᵐ).
15. Regression: r is linear-only; extrapolation beyond x-range is unsupported; r² not "accuracy of prediction for individuals".
16. Keep "standardizing data" and "normal approximation to binomial" as two separate problems.
17. Outliers: check mean/median gap and boxplot fences; report robust statistics when skew is real.
18. Randomization enables causation; association from observational studies comes with confounder caveats.