Linear Separation

Coordinate Space

Let's talk about what we're even doing here. We're graphing a line along a 2 dimensional space. On the web, the 2 dimensional space is pixels and it's tied to the origin of the canvas, up in the upper left.

Mapping the origin to the center

But what we want, when doing math is an origin that's in the middle. With a top mark being 1 and a bottom area being -1. You can expand this to whatever multiple you want, 10, 100, 1000. It's all the same. The area from -1 to 1 is an infinite amount of decimals.

We can convert our screen pixel to our orthogonal coordinate plane using a built in function in p5.

In index.js:

const minVal = -1
const maxVal = 1

class Point {
  constructor(x, y) {
    this.realX = x
    this.realY = y
    this.x = map(x, minVal, maxVal, 0, width)
    this.y = map(y, minVal, maxVal, height, 0)
  }
}

Map from p5

map in p5 isn't specific to coordinates and it isn't anything like the javascript Array.map(). It's just a normal function that takes a number as it's first argument. It's next 2 arguments are the min and max range we should start with. In this can we're saying -1 to 1. The next 2 arguments are the scope we want to covert to. So in this case, the left and right edge of our canvas. So map can be thought of as a converted from one range to another.

Draw the coordinate plane

Let' give us some cross-hairs so we see our new coordinate structure.

function drawCoords() {
  strokeWeight(1)
  stroke(0)
  x1 = map(minVal, minVal, maxVal, 0, width)
  y1 = map(0, minVal, maxVal, height, 0)
  x2 = map(maxVal, minVal, maxVal, 0, width)
  y2 = map(0, minVal, maxVal, height, 0)
  line(x1, y1, x2, y2)
  x1 = map(0, minVal, maxVal, 0, width)
  y1 = map(minVal, minVal, maxVal, height, 0)
  x2 = map(0, minVal, maxVal, 0, width)
  y2 = map(maxVal, minVal, maxVal, height, 0)
  line(x1, y1, x2, y2)
  stroke(40)
  for (let i = -1; i < 1; i += 0.05) {
    x1 = map(-0.03, minVal, maxVal, 0, width)
    y1 = map(i, minVal, maxVal, height, 0)
    x2 = map(0.03, minVal, maxVal, 0, width)
    y2 = map(i, minVal, maxVal, height, 0)
    line(x1, y1, x2, y2)
  }
  for (let i = -1; i < 1; i += 0.05) {
    x1 = map(i, minVal, maxVal, 0, width)
    y1 = map(-0.03, minVal, maxVal, height, 0)
    x2 = map(i, minVal, maxVal, 0, width)
    y2 = map(0.03, minVal, maxVal, height, 0)
    line(x1, y1, x2, y2)
  }
}

And add the function to our draw() function.

What is a line anyway?

We're going to talk about a line for a second. Because you need to understand this to understand how a perceptron understands the world. What weights even are. So come with me.

A line in terms of analytic geometry is a way to describe how the line would look, how it's pointing on a plane. Technically a line goes on forever in both directions, so it's not quite a segment, which would need 4 coordinates to define it.

Everytime we draw a line in p5 we're actually just drawing a segment. A line, only needs to describe the relationship between x and y.

So we need to know at what point does the line cross the Y axis when X is 0. It's represented as.

y = m * x + b

Where b is the Y number when the line crosses 0 for X, and m is the slope of the line, the angle, the rise over run, if you will. So let's create a few lines and we'll see what I mean.

If we go to the f(x) function and made up some values for m and b we can really see what a line is. If m, which remember, is the slope, rise, meaning the number of units it'll move up for every unit it moves to the right and b is the value of Y when X is equal to 0.

So let's create some examples to understand how it's working.

function setup() {
  m = -2 / 1
  b = 3
}

The job of the weights

The weights in a perceptron actually fit into this equation quite well and create a line.

The perceptron's job

THe perceptron, according to the guy who invented/discovered them in mathematics, are quote, "only capable of solving linearly solvable problems". That means, if you can graph it and split it with a line, you can use a perceptron to solve it. Otherwise, you have to add additional layers of perceptrons to get a good result.

And that's what we're going to do in the next section.