Point Estimation: MLE and Method of Moments Explained

Point estimation is the task of producing a single best-guess number for an unknown population parameter: one value for μ, one for σ², one for p. The sample mean x̄ = 52.3 is a point estimate of the population mean; the sample proportion p̂ = 0.4 is a point estimate of the true proportion.

It matters because decisions need numbers, not ranges. A factory must set its machines to one target weight, a pollster must report one figure, and an insurer must pick one premium. Point estimation is the theory that tells us which single-number formulas are sensible and which are systematically misleading.

Key Concepts

• Estimator vs estimate: the estimator is the formula (e.g. X̄); the estimate is the number it produces from your data (e.g. 52.3).
• Unbiasedness: E(estimator) = parameter. X̄ is unbiased for μ; the sample variance with divisor n-1 is unbiased for σ², while dividing by n gives a biased (too small) value.
• Maximum likelihood (MLE): choose the parameter value that makes the observed data most probable. For a normal population, the MLE of μ is x̄; for a proportion, it is p̂.
• Method of moments: set population moments equal to sample moments and solve. Setting μ = x̄ estimates the mean; setting μ² + σ² = average of squares estimates the variance.
• Consistency and efficiency: a consistent estimator converges to the truth as n grows; among unbiased estimators, the efficient one has the smallest variance.

How It Works

MLE in four steps: 1. Write the likelihood L(θ), the joint probability of the data. 2. Take the log to turn products into sums. 3. Differentiate and set to zero. 4. Solve for θ̂. Method of moments in three steps: 1. Write the first k population moments in terms of the k parameters. 2. Replace them with the sample moments. 3. Solve the equations. For example, for an exponential population, E[X] = 1/λ, so λ̂ = 1/x̄ by moments, which here matches the MLE.

Worked Example

A coin is flipped 100 times with 60 heads. Estimate p by MLE. The likelihood is L(p) = C(100,60) p⁶⁰(1-p)⁴⁰. Log-likelihood: constant + 60 ln p + 40 ln(1-p). Derivative: 60/p - 40/(1-p) = 0, giving 60(1-p) = 40p, so 60 = 100p and p̂ = 0.60, the sample proportion, as expected. The method of moments gives the same 0.60, so both methods agree.

Common Misconceptions

• MLE is not always unbiased: the MLE of the normal variance divides by n and underestimates σ² on average; the unbiased version divides by n-1. "Most likely" and "right on average" are different virtues.
• A point estimate is not the truth: p̂ = 0.60 does not mean p is exactly 0.60; it is the centre of our uncertainty. Always pair it with a standard error or interval.
• Method of moments can give silly answers: for a uniform(0, θ) population it can produce estimates below the largest observation, which is impossible. MLE respects such boundaries better.

Real-World Applications

• Manufacturing: estimating the true defect rate p from a sample to decide whether a production line needs recalibration.
• Finance: estimating average returns and volatility of a stock from historical prices for risk models.
• Medicine: estimating a drug's cure rate from trial data before regulators approve it.

Point estimation picks the single best number for a parameter, with MLE maximising the data's probability and the method of moments matching sample moments to population ones. Judge estimators by unbiasedness, consistency and efficiency.