Starting with the Perceptron

The Perceptron

The simplest unit in a neural network is the perceptron. The Perceptron is the part of the neuron that makes a decision to fire and send an electric signal down the axion to another neuron in the brain.

In our brain this happens when the chemical signals it receives from it's dendrites, add up and at a certain level, or chemical calculation. If they do, it fires, if they don't, it doesn't fire. That's it. 2 choices, fire and send data along to other layers of neurons or don't.

In javascript, let's code one out, then talk a bit about it:

class Perceptron {

  constructor(size = 2) {
    this.weights = new Array(size)
    for (let i = 0; i < this.weights.length; i++) {
      this.weights[i] = random([-1, 1])
    }
  }

  static random = (items) => {
    return items[Math.floor(Math.random()*items.length)];
  }
}

Weights

We can think of weights as the strength of the connection between the neuron and it's input. We can think of the input as the retina on your eye. When light hits it, it is set to that light's data number. Each neuron can have multiple inputs and each input get's it's own weight.

Weights are adjusted and the perceptron learns which guesses were correct and which were wrong. It will increase the weight on the inputs that led to the correct guess.

This is why we need training data that is correctly labeled. So the computer knows right from wrong.

To start, we just need to initializing a new perceptron with randomized weights, 1 or -1 .

Activation Function

We need a way to decide if the neuron fires or not. This is called the activation function.

// Activation function
const sign = (num) => num >= 0 ? 1 : -1

Inside the class:

guess(inputs) {
  let sum = 0
  for (let i = 0; i < this.weights.length; i++) {
    sum += inputs[i] * this.weights[i]
  }
  const output = sign(sum)
  return output
}

So now in our setup function, we can create some inputs and make a guess.

const setup = () => {
  const inputs = [-1, 0.5]
  const p = new Perceptron(inputs.length)
  const guess = p.guess(inputs)
  console.log(guess)
}