Supervised Learning: Regression and Classification

Supervised learning trains on labeled examples — each one comes with the correct answer. Thousands of house listings labeled with sale prices teach price prediction; thousands of emails labeled spam or not-spam teach classification. The "supervision" is simply the presence of correct answers during training.

This is the most-used branch of machine learning because it maps directly onto measurable questions: how much will this cost, which category is this, will this customer leave? Historical examples with known outcomes become predictors of the future.

Key Concepts

• Labeled data: training examples paired with their correct answers.
• Features (X): the input measurements, such as a house's area and rooms.
• Target (y): the value to predict — a number for regression, a category for classification.
• Regression: predicting a continuous number, like price or temperature.
• Classification: predicting a category, like spam/not-spam.
• Loss function: a formula measuring prediction error; training minimizes it.
• Train vs test set: the model learns on training data and is honestly evaluated on separate unseen data.
• Overfitting: memorizing training quirks so test performance suffers.

Formulas

• Mean squared error (regression loss): MSE = (1/n) times the sum of (y − prediction) squared. Averaging squared gaps punishes big mistakes more. Units: the square of the target's units.
• Accuracy (classification score): accuracy = correct predictions divided by total predictions. Simple, but misleading when classes are imbalanced.
• Assumption: labels are correct and future data resembles training data; when the world changes, performance drops.

Worked Example

A regression model predicts three house prices (lakh rupees): true 10, 20, 30 versus predicted 12, 18, 33. Errors: −2, 2, −3. Squared errors: 4, 4, 9. Sum = 17. MSE = 17 divided by 3, about 5.67. A spam filter tested on 200 emails gets 184 right: accuracy = 184 divided by 200 = 92 percent, error rate 8 percent. Measure the gap between prediction and truth, average it, shrink it — the daily bread of supervised learning.

Common Misconceptions

• Regression and classification are unrelated. Both are supervised learning; they differ only in the target type.
• High training accuracy means a good model. It can mean memorization; only test performance proves generalization.
• Classes must always be balanced. Natural imbalance (fraud is rare) is fine with class weights and the right metric.

Real-World Applications

• Price prediction: houses, taxi fares, energy demand.
• Spam detection: sorting email into spam and inbox.
• Disease diagnosis: predicting illness from symptoms and tests.
• Credit scoring: default risk and expected loss guide lending.

Supervised learning learns from labeled examples: regression predicts numbers, classification predicts categories, and training minimizes the gap between predictions and truth. Evaluate on unseen test data, watch for overfitting, and match metrics to the real cost of mistakes.