Calculating the Error

Let's make a new function in our perceptron. It's going to train.

train(inputs, target) {
  const guess = this.guess(inputs)
  const error = target - guess
  for (let i = 0; i < this.weights.length; i++) {
    this.weights[i] += error * inputs[i]
  }
}

Now we're adjusting the weight value on this perceptron by the difference between the target and the guess. But we don't want to steer the full way towards the target. So we need another factor in this equation. The learning rate.

Learning Rate

Tells the perceptron how little to move the final correction. We'll have many samples to choose from, so steering all the way there won't be as useful as steering partly towards the goal and allowing other guesses to move us the rest of the way there.

So let's add a small learning rate to the end of this multiplication.

train(inputs, target) {
  const learningRate = 0.1
  const guess = this.guess(inputs)
  const error = target - guess
  for (let i = 0; i < this.weights.length; i++) {
    this.weights[i] += error * inputs[i] + learningRate
  }
}

This will make 1/10 of the correction needed.

Bias

With all this multiplication, zeros really muck up the works. Resulting in a zero calculation. So we add another variable, called a Bias. A bias is the frequency with which a neuron will fire even when it's under the threshold for firing. So it can be thought of as a built in weight that's equal to 1 and randomly gets neurons to fire based on it's weight. Sort of like real bias.

Let's add the bias to each point. It'll be the same value, 1.

class Point {
  constructor(x, y) {
    this.realX = x
    this.realY = y
    this.bias = 1
    ...

Let's add a new weight to the mix. This will be our bias. The bias is an additional piece of data that goes along with the inputs.

In checkBrain() we need to pull the bias off each point.

function checkBrain() {
  let right = (total = wrong = 0)
  points.forEach(point => {
    const inputs = [point.realX, point.realY, point.bias]
    const target = point.label
    const guess = neuron.guess(inputs)

In the Perceptron class:

constructor(size = 3) {
  this.weights = new Array(size)
  ..

guessY(x) {
  const w0 = this.weights[0]
  const w1 = this.weights[1]
  const w2 = this.weights[2]
  return -(w2 / w1) - (w0 / w1) * x
}

In the train function:

function train() {
  const point = points[currentPoint]
  const inputs = [point.realX, point.realY, point.bias]
}

Next: Linear Separation