Neurons as Weighted Sums

Goal

By the end of this lesson, you can compute a neuron's weighted sum, explain what each weight and the bias do, and predict the effect of changing one parameter.

One neuron is a small arithmetic rule

Suppose a neuron receives two inputs:

x1 = 2, x2 = 3

and has:

w1 = 0.5, w2 = -1, b = 4

The pre-activation value is:

z = w1*x1 + w2*x2 + b

Substitute the numbers:

z = 0.5*2 + (-1)*3 + 4

z = 1 - 3 + 4 = 2

Nothing mysterious happened. The neuron multiplied each input by a weight, added the contributions, and then added a bias.

What each parameter changes

A weight controls how strongly an input contributes and in which direction.

If x1 is positive and w1 increases, the term w1*x1 increases.

A negative weight reverses direction: when a positive input grows, a negative weighted contribution becomes more negative.

The bias is an additive offset. It can move the result even when all input values are zero.

This is why it helps to print contributions separately when debugging:

• w1*x1
• w2*x2
• b
• final sum

Trace the Lab one term at a time

The Lab uses:

x = np.array([2.0, -1.0])
w = np.array([0.5, 2.0])
b = 1.0

1. Click Run and verify contributions: [1.0, -2.0], bias: 1.0, and weighted sum: 0.0.
2. Find the first weight in w and change only 0.5 to 1.0:

w = np.array([1.0, 2.0])

3. The matching first input is 2.0. Increasing its weight by 0.5 should increase the first contribution by 2.0 × 0.5 = 1.0.
4. Before running, predict the new first contribution and the new weighted sum.
5. Click Run and confirm the first contribution becomes 2.0 and the weighted sum becomes 1.0 while the second contribution and bias stay fixed.
6. Restore the first weight to 0.5.

Loading lab…

After the guided pass, change only the second weight and make the same kind of hand prediction before running.

If your final answer is wrong, stop at the first contribution that differs from your hand calculation. That is more useful than changing several values until the result looks right.

Order is part of meaning

Suppose the inputs are [temperature, humidity] but you accidentally pair them with weights learned for [humidity, temperature].

The arithmetic still runs. The meaning is wrong.

This is an important class of ML bug: a computation can be numerically valid while semantically mismatched.

Later, matrix multiplication will perform many weighted sums at once. The same requirement remains: each position must correspond to the feature the parameter was learned for.

Technical bridge: dot products

For vectors x and w, the weighted sum is often written as a dot product:

z = x · w + b

Matrix multiplication is how a layer computes many such dot products efficiently for many examples and neurons.

Work one neuron from input to output

Suppose a neuron receives two features:

x1 = 3
x2 = 2

with weights w1 = 0.5, w2 = -1.0, and bias b = 1.

Before any activation function, the neuron computes:

z = 0.5×3 + (-1.0)×2 + 1
  = 1.5 - 2 + 1
  = 0.5

Each weight answers a local question: how strongly and in which direction should this feature affect the pre-activation value?

A positive weight pushes z upward when its input increases. A negative weight pushes z downward.

The bias is different: it shifts the result even when all input values are zero.

Do not interpret one learned weight as a complete explanation of the model. In a network, later layers combine many neuron outputs, so the meaning of one weight depends on the whole computation path.

Quick Check

1. What does a negative weight do when its positive input increases?

It increases the sum less strongly than a positive weightIt makes that input contribute a more negative amount to the weighted sumIt affects the activation after the nonlinearity, not the weighted sum itself

2. What is the bias?

A scale factor that multiplies every input weightThe average prediction error added after the loss is computedAn additive offset independent of current input products

3. Best first debug step for a wrong weighted sum?

Check each input × weight product and the bias one by oneRefit with a different random seed before checking the current calculationChange the activation function before checking the arithmetic

0 of 3 questions answered.

Key Takeaways

• A neuron begins with a weighted sum plus bias.
• Weights control input influence and direction.
• Bias shifts the result independently of current input products.
• Feature/weight ordering is part of the computation's meaning.
• Inspect individual contributions before debugging a final sum.

Next Lesson

Next, you will transform these weighted sums with activation functions so stacked layers can represent nonlinear patterns.

References

• PyTorch, torch.nn.