HoCo Hour of Code

Deep learning and machine learning are often seen as the same thing. In reality, they are two things that work very differently.

Deep learning is a type of machine learning that is focused on training artificial neural networks and is able to become progressively smarter (learn based on its own experiences). The neural networks used in deep learning are actually inspired by the human brain! Neural networks contain layers of neurons, called hidden layers. These hidden layers are used to perform hidden transformations on data.

Before proceeding further, it is important to look at some important machine-learning math concepts.

A function defines the relationship between some inputs, called the domain, and their outputs, called the range. Functions are usually denoted with the equation of y=f(x), where the function takes in some input, x, and gives an output, y.

A derivative is the slope of a function at a given point. The derivative of a function at point x, is usually denoted as f'(x). And as a reminder, the slope is yx (rise over run).

Neural networks make it easier to work with more complex relationships such as non-linear ones (quadratic, cubic, etc). Many real-world scenarios typically don't have a linear solution, usually, they are more complex quintic or sextic functions. Essentially neural networks can create higher-degree functions in order to give more accurate outputs.

The most basic form of a neural network is the perceptron. It is a linear machine learning algorithm for the purpose of binary classification (classifying input into one of two classes).

The linear function, y=mx+b, is similar to what a perceptron is doing. In the image above, the importance of the inputs x1, x2, and x3 in the final output are determined by their respective weights w1, w2, and w3 assigned to the input. The formula for the output is x1w1+x2w2+x3w3. If the output is above a threshold value, it will result in a 1, else it will result in a 0 (typical binary classification).

Obviously, this simple model is not ideal and is often too simple to use in real-world applications. A better model would use multiple layers of neurons to determine an output. Enter neural networks, which have an input layer, hidden layer(s), and an output layer. The input layer represents the dimensions of the input vector (or list). The hidden layer consists of nodes that divide the input into different regions. Think of the hidden layer as a function that transforms an input into a given output (like y=f(x)). The output layer is the output of the network.

In Deep Learning Part 2, we will learn more about how neural networks work.