Limit Theorems & Asymptotics

Limit theorems describe the behavior of random variables and their sums as sample sizes increase, forming the foundation for many statistical inferences.

Key theorems include:

• Law of Large Numbers: As sample size increases, the sample mean converges to the true mean.
• Central Limit Theorem: The sum of many independent random variables approximates a normal distribution, regardless of their original distributions.
• Delta Method: Approximates the distribution of a function of asymptotically normal random variables.
• Large Deviation Theory: Studies the probability of rare events in the asymptotic regime.

These theorems explain why many machine learning methods work well with large datasets, provide theoretical guarantees for statistical estimators, and justify approximation methods used in complex models.