Supervised Learning: Learning a Mapping from Examples

Supervised learning begins with examples whose desired answers are known.

The model adjusts parameters so inputs map to those targets, then applies the learned relationship to new cases.

Supervised learning needs representative input-output examples and a target that matches the real decision.

A model can optimize the wrong label perfectly and still be useless.

A concrete example: house prices

Historical rows contain:

floor area,
neighborhood,
bedrooms,
age,
sale price.

The first four become features, and sale price is the target. The model learns patterns that estimate a new home's likely price.

Training examples

Each example includes:

(input features, known target)

Examples may come from:

historical operations,
human labeling,
experiments,
sensors,
simulations,
or expert rules.

Their collection process affects what the model can learn.

Classification

Classification predicts a category:

spam or not spam,
fraud risk class,
image label,
support topic.

The model often produces scores or probabilities before a decision rule selects a class.

Regression

Regression predicts a number:

price,
demand,
delivery time,
energy use.

The target's scale and error consequences determine the loss and evaluation metric.

Multi-class and multi-label

Multi-class classification selects one among several categories.

Multi-label classification allows several:

photo contains beach and person,
ticket concerns billing and account access.

The label design must reflect whether categories are exclusive.

Features

Features are information available at prediction time:

measurements,
categories,
text,
images,
histories,
derived values.

Feature quality and availability can matter more than algorithm novelty.

Target definition

The target must match the intended outcome.

Predicting "clicked" can reward sensational content when the real goal is satisfaction. Predicting "was hired" can reproduce historical hiring bias rather than job success.

Target choice is a product and ethical decision.

Model

A model is a parameterized function.

Examples include:

linear regression,
decision tree,
gradient-boosted ensemble,
neural network.

Different model families express different relationships and trade interpretability, data need, and compute.

Loss function

Training minimizes a loss measuring prediction error.

Examples:

squared error for numbers,
cross-entropy for categories,
custom weighted loss for costly mistakes.

The loss tells the optimizer what "better" means during training.

Optimization

An optimizer adjusts parameters to reduce loss.

Gradient-based methods use how loss changes with each parameter. Training repeats over examples for several passes.

Lower training loss does not guarantee better real-world performance.

Generalization

Generalization means performing well on new examples from the intended environment.

Validation and test sets estimate it. If deployment data differs from training data, even honest held-out performance can mislead.

The goal is not memorizing the dataset.

Representative data

Training data should cover:

users,
devices,
regions,
seasons,
rare cases,
and operating conditions

expected in use.

Missing groups create blind spots.

Sampling bias

Data may overrepresent easy-to-collect cases.

A medical model trained only on one hospital or device may fail elsewhere. A support model trained only on resolved tickets misses abandoned customers.

Document the population and collection process.

Label quality

Labels can be:

noisy,
subjective,
delayed,
inconsistent,
or produced by another imperfect system.

Measure reviewer agreement, define guidelines, and preserve uncertainty where the task is ambiguous.

Class imbalance

Rare outcomes such as fraud may be a tiny fraction.

A classifier predicting "not fraud" for everything has high accuracy but no value.

Use precision, recall, ranking metrics, weighting, sampling, and threshold design.

Data leakage

Leakage occurs when training inputs reveal information unavailable at prediction time.

Examples:

future collection status for default prediction,
post-outcome note,
target-derived aggregate.

Leakage creates impressive tests that collapse in deployment.

Baselines

Compare against simple alternatives:

always predict average,
current business rule,
logistic regression,
previous model.

A complex model should earn its operational cost through meaningful improvement.

Calibration

A calibrated probability of 0.8 should correspond to roughly 80 percent occurrence among similar predictions.

Calibration matters when scores drive:

pricing,
capacity,
review priority,
or risk thresholds.

Ranking quality alone may not provide trustworthy probabilities.

Decision threshold

The model score becomes an action through a threshold.

Changing the threshold trades false positives and false negatives. Product cost, human capacity, and risk determine the operating point.

The threshold is part of the deployed system.

Deployment feedback

Model decisions can change future data.

A fraud model blocks transactions, so labels are observed mainly for allowed transactions. A recommender changes what users see and click.

Monitor feedback loops and design evaluation deliberately.

Treat the prediction as a product contract

A deployed supervised model does not merely return a number. It participates in a contract:

which population may be scored,
which features must be available,
how fresh those features must be,
what the output means,
which threshold or policy consumes it,
and what happens when confidence is low.

For example, a late-payment probability might trigger a reminder, a manual review, or a credit restriction. Those are different products even if they use the same model score. Document the intended action and prohibited uses so a convenient prediction is not quietly reused for a decision it was never evaluated to support.

Knowledge check

How do classification and regression differ?
Why is target definition a product decision?
What does generalization mean?
How can class imbalance make accuracy misleading?
What makes a feature leak future information?

The one idea to remember

Supervised learning fits a mapping from features to known targets through labeled examples. Its real quality depends on target meaning, representative data, trustworthy labels, leakage prevention, held-out evaluation, and a decision threshold matched to consequences.