Decision Boundaries

Goal

By the end of this lesson, you can explain a decision boundary, probe a two-feature classifier near that boundary, and connect a small input change to a possible class change.

A classifier divides feature space into regions

Imagine a map where every location is colored according to the class a model would predict there.

One region might be colored “pass” and another “fail.”

The border where the color changes is the decision boundary.

For a classifier with two features, you can often picture this directly. A point on the map represents one pair of feature values. Move the point, and you may eventually cross into a region that receives a different class.

Why points near the boundary deserve attention

Suppose one example receives positive-class probability 0.49 under a 0.5 decision threshold.

A small input change moves the probability to 0.51.

The numerical score changed only a little, but the final class flipped.

That makes near-boundary cases useful for debugging. They reveal where small measurement changes, noise, or policy choices can change the decision.

Near-boundary does not automatically mean “wrong.” It means the example is close to the model's current dividing line.

The boundary for two-feature logistic regression

With two features, logistic regression forms:

z = w1*x1 + w2*x2 + b

At a 0.5 probability threshold, the class changes where z = 0.

That equation describes a straight line in a two-dimensional feature space.

The signs and sizes of the coefficients influence how that line is oriented, but the most useful beginner skill is simpler: change one feature, watch the model score, and see whether the point crosses the decision border.

Probe the learned boundary

The Lab fits logistic regression using two features: study hours and attendance.

1. Click Run.
2. Read probe probabilities: [0.408, 0.525, 0.692] and probe labels: [0, 1, 1]. Each probe is [study hours, attendance].
3. The middle probe, [5.2, 0.79], is closest to 0.5, so it sits near the boundary.
4. In probe_points, change only its study hours: [5.2, 0.79], becomes [5.0, 0.79],.
5. Before running, predict: is 0.2 fewer study hours enough to cross the boundary?
6. Click Run. The middle probability drops to 0.466 and its label flips from 1 to 0. A small move near the boundary changed the decision.
7. Press Reset. Now change only the attendance of the same probe: [5.2, 0.79], becomes [5.2, 0.75],. Run again. The probability barely moves (0.524) and the label stays 1. In this fitted model, a small attendance change matters much less than a small study-hours change.
8. Press Reset afterward.

Loading lab…

The useful evidence is the before/after pair: which feature changed, how the model score moved, and whether the final class switched.

A neat boundary can still be unreliable

A straight line can look clean and confident while being based on poor evidence.

For example, the training data might omit an important subgroup or contain very few examples in one region. The model will still draw a boundary there because it must make a prediction, even when it has little support from data.

So when a boundary looks surprising, ask:

• Is this point inside the range represented by training data?
• Is an important feature missing?
• Are labels reliable near this region?
• Does behavior remain similar across validation folds?

More flexible models can draw curved or fragmented boundaries. That flexibility can fit more complicated patterns, but it can also make overfitting easier.

A common misconception

“A point near the boundary is uncertain in exactly the same way a human would be uncertain.”

The boundary describes the model's decision geometry. It does not automatically represent human uncertainty, data quality, or real-world ambiguity.

Quick Check

1. What is a decision boundary?

The average predicted probability over the datasetThe border in feature space where the predicted class changesThe parameter value where training loss is smallest

2. Why are near-boundary examples useful to inspect?

They contain no predictive information because their scores are near the thresholdA small change in their features can flip the predicted classThey must be mislabeled because the model is uncertain

3. For two-feature logistic regression, what shape is the 0.5 boundary?

A circle whenever both features are scaledA curved boundary unless the intercept is exactly zeroA straight line, where the weighted score equals zero

0 of 3 questions answered.

Key Takeaways

• A decision boundary separates regions assigned different predicted classes.
• Near-boundary examples can flip class after small input or policy changes.
• Two-feature logistic regression has a linear decision boundary at the 0.5 threshold.
• Boundary shape is model behavior, not proof that the data or decision is trustworthy.

Next Lesson

Next, you will look inside classification errors so accuracy does not hide which kind of mistake the model is making.

References

• scikit-learn, LogisticRegression.