Regression Coefficients: Meaning, Formula & Examples

A regression coefficient tells you how much the dependent variable changes, on average, when the independent variable increases by one unit. In the regression line of y on x, written y = a + b.x, the number b is the regression coefficient of y on x (b_yx): if b = 2.5, every extra unit of x raises the predicted y by 2.5 units on average. It is the slope of the best-fitting straight line through the data.

Regression coefficients turn scattered data into usable predictions: economists forecast demand from price, doctors estimate risk from measurements, and Class 11-12 statistics students meet them in every exam. Unlike the correlation coefficient, a regression coefficient has real units and a direct per-unit meaning that decision-makers can act on.

Key Concepts

• Two regression lines: y on x (y = a + b_yx.x) minimises vertical errors; x on y (x = a' + b_xy.y) minimises horizontal errors. Both pass through the mean point (x-bar, y-bar).
• Formulas: b_yx = r.(sigma_y / sigma_x) and b_xy = r.(sigma_x / sigma_y), where r is Karl Pearson's correlation coefficient.
• Product property: b_yx x b_xy = r^2, so r = +/- sqrt(b_yx.b_xy), with the sign matching the coefficients.
• Sign rule: both coefficients always carry the same sign as r.
• Size: if |r| = 1 the two lines coincide; if r = 0 they are perpendicular (y = y-bar and x = x-bar).
• Units: b_yx carries (units of y)/(units of x), e.g. cm per year for height on age.

Formulas

• y on x: b_yx = r x (sigma_y / sigma_x); line: y - y-bar = b_yx.(x - x-bar).
• x on y: b_xy = r x (sigma_x / sigma_y); line: x - x-bar = b_xy.(y - y-bar).
• Product rule: b_yx x b_xy = r^2.
• Angle between lines: tan(theta) = ((1 - r^2)/|r|) x (sigma_x.sigma_y)/(sigma_x^2 + sigma_y^2).
• Assumption: an approximately linear relationship; the coefficient is meaningless for strongly curved data.

Worked Example

Given x-bar = 10, y-bar = 20, sigma_x = 4, sigma_y = 5, r = 0.8.

1. b_yx = 0.8 x (5/4) = 0.8 x 1.25 = 1.00.
2. b_xy = 0.8 x (4/5) = 0.8 x 0.80 = 0.64.
3. Check: 1.00 x 0.64 = 0.64 = r^2 = 0.64. It matches.
4. Line of y on x: y - 20 = 1.00(x - 10), i.e. y = x + 10.
5. Predict y for x = 14: y = 24. Each extra unit of x adds 1.00 unit to y on average.

Common Misconceptions

• A big coefficient means strong correlation. Not necessarily: b_yx = r.(sigma_y/sigma_x) mixes correlation with spread; a steep slope can come from a large sigma_y with weak r.
• Regression proves causation. It only describes association; ice-cream sales and drowning both rise in summer without causing each other.
• There is only one regression line. There are two, y on x and x on y; they coincide only when |r| = 1. Using the wrong one biases predictions.

Real-World Applications

• Economics: demand forecasting, salary on experience.
• Medicine: dose-response curves, growth charts.
• Engineering: instrument calibration curves.
• Machine learning: linear regression underlies supervised learning.

A regression coefficient is a slope with a job: it converts x goes up into y goes up by this much, on average. Remember the two lines, the product rule b_yx.b_xy = r^2, and never read causation into a slope.