Neural Network and Deep Learning Foundation - Introduction to Neural Network

The neural network and deep learning are complex subjects. That’s why it is crucial to understand them at an abstract level properly before you create your models.

Knowing the theories behind the neural network and deep learning will help you to create accurate models. Not only that, but it will also provide you the ability to solve complex deep learning problems.

A large part of this article will be theory-based. So, if you are entirely new to the world of neural networks and deep learning, then this article is for you.

Also, if you prefer to learn about how neural network and deep learning work through videos, then make sure to check out the following course:

Artificial Intelligence Foundations: Neural Networks

Okay then, let’s get started.

What is Deep Learning?

Deep learning is the subset of machine learning that uses neural networks to emulate the human brain and its intelligence.

One of the purposes of a deep learning model is to process complex data and identify patterns. As a result, it can make its own decisions.

In contrast to machine learning, deep learning relies on layers of neural networks. Don’t worry about neural networks for now. We will get to that in a bit.

AI vs Machine Learning vs Deep Learning

Before we talk more about a neural network and deep learning, let’s draw a line between AI, machine learning & deep learning to make the terms clear.

Artificial Intelligence – Artificial intelligence is the theory of machines imitating human intelligence to perform tasks. For example, vision, speech recognition, making decisions, etc.
Machine Learning – Machine learning is the subset of AI that allows the machine to learn without being explicitly programmed. Think of machine learning as an application of AI where it learns semi-automatically from data.
Deep Learning – Deep learning is a subset or family of machine learning. Artificial neural networks are the building blocks of deep learning that imitate the functions of the human brain.

In short, machine learning & deep learning both fall under the umbrella of artificial intelligence.

The Biological Neuron

Learning how biological neuron works is essential when it comes learning about the neural network and deep learning. So, in this section, we will discuss how they work.

First of all, let’s not get overwhelmed by the picture since not every aspect of the diagram is essential.

However, we will only focus on the following parts:

Dendrites
Nucleus
Axon

So think of dendrites as the inputs passing down to the nucleus. And then axons as the outputs.

In a much abstract perspective, our neuron will accept some inputs to perform some metrics and calculations. After that, the neuron is outputting the results as a single value.

Thus the result we get from a single neuron can also work as an input for other neurons.

Now the question arises that how can we take biological neurons and represent them through machines? That’s where a perceptron model comes into play.

What is a Perceptron Model?

A perceptron model is the representation of a biological neuron mathematically in an artificial neural network.

Let’s try converting a biological neuron into a perception model.

If you can recall, we talked about dendrites being the inputs and the axons as the outputs. In the middle, we have the nucleus in which we perform some calculations.

Now, if we replace the biological terms with mathematical ones, we get a basic perception model. Here’s what it looks like:

So we have two inputs x1 and x2, going into this perception model. And then, the input values go through a function f(x) to give us the output y.

The f(x) can be any function or calculation. For example, it can even be y = x1 + x2.

Now obviously, these neural network functions are most likely to be more complicated and advanced.

Weights

Let’s talk about weights. So, what are those?

Weights are parameters within the perceptron model that transforms the input value. It helps us to adjust the values for the perceptron to learn.

Weights work by multiplying against inputs or x.

Observe the following diagram:

After adding the weights, our equation y = x1 + x2 becomes y = x1w1 + x2w2.

We can now adjust the weight until we get the desired output value of y.

A neural network is the product of multiple perceptrons. We will get to that in a bit when we discuss artificial neural networks.

And these weights allow each perceptron or neuron in a neural network to adjust its output before passing it into the next one.

But what happens when we have the x as 0? Adjusting the weight won’t change anything in this case. That’s where bias comes into play.

Bias

Bias is an additional parameter that also allows us to adjust the output. When x 0, it does not matter even if you add the weight since it will always end up being 0. As a result, adding the bias to the input will prevent that.

It’s similar to the linear equation y = mx + b, where the b is constant.

Here we have a similar situation as the b provides a way to fit the input data and contributes to the output.

After adding the weights and bias, our equation becomes y = (x1w1 + b) + (x2w2 + b).

Hopefully, the concept behind the perceptron model is clear to you. Think of it as a simple replication of a biological neuron.

Next, we will see how these perceptrons work together to form a neural network.

What is a Neural Network?

By definition, a neural network is a set of algorithms modeled after the biological neurons of a human brain. You can consider them to be the heart of any deep learning model designed to detect and classify patterns.

But what makes a neural network are the perceptron layers or the perceptrons. In other words, multiple perceptrons are connected in a network using the multi-layer perceptron model.

Let’s take a moment to understand the following diagram.

Here you can see that we have vertical layers of perception models, and their outputs are acting as inputs for the next layer of perceptions.

In short, each layer feeds the output of their perceptrons into the perceptrons of the next layer.

The first layer is the input layer which receives some data, while the last layer is the output layer. Also, the layer at the end can consist of multiple perceptrons or neurons rather than a single one.

Then we have the layers in between the input layer and the output layer. These layers are known as hidden layers. For this illustration, we only have one hidden layer. But it can be multiple.

One of the responsibilities of the hidden layers is to convert the inputs into values that the output layer can understand. And if we have two or more hidden layers, our neural network becomes a deep neural network.

So, a deep neural network is a neural network that has two or more hidden layers. For example:

To understand a neural network and deep learning enitrely, it is also crucial to know their benefits.

So, here are some of the benefits of an artificial neural network:

Neural networks can take data and have the ability to learn on their own.
The output of a neural network is not limited to the input it receives.
They can uncover and detect patterns from unseen data and make predictions.
Neural networks can process data that are not sequential. In other words, they can work with non-linear data.
One of its main benefits is also fault-tolerance. If any part of data goes missing, it does not affect its operational ability to process data.
Neural networks can learn from examples and use that learning experience to detect similar events.

How Does a Neural Network Work?

We already know that a neural network is a network of neurons, where we have the input layer, hidden layers, and an output layer. And we saw that the neurons are organized based on layers.

So the input layer takes in data while the output layer gives us results. The hidden layers are where the magic happens. In other words, layers that are between the input layer and the output layer solves the problem.

Each neuron and the edge connecting them holds a number.

The input neurons will propagate their numbers to the neurons of the next layer by taking the sum of products of all input neurons and the corresponding edge numbers or weights. The process of propagating from the input layer until the output is known as the forward propagation.

And then, we take the values of the last layer and compare them to the labels of known data or expected answers. Depending on that, we keep updating the weights starting from the output layer till the first. This process is known as backward propagation.

Through these propagations, we run multiple input values and weights through a neural network. As a result, it learns how to detect patterns and solve problems.

The concepts behind propagations are more complex, which is beyond the scope of this article. However, to understand the foundation of a neural network and deep learning, I had to provide a basic explanation.

Now, what kind of problems can a neural network solve? It depends. A problem can either be complex or straightforward. Let’s say, for example, an email filtering system uses neural networks to identify spam emails.

The neural network in a self-driving car is likely to be more complicated. As a result, it needs more hidden layers and advanced functions to solve problems.

Application of Neural Network and Deep Learning

Here are some of the most notable applications of neural network and deep learning:

Image Recognition: Neural networks are used extensively in image recognition applications, such as facial recognition and object detection. By training a neural network on a large dataset of images, the network can learn to identify specific patterns and features in images and use that information to classify new images.
Natural Language Processing: Neural networks can be used to analyze and understand human language, enabling applications such as voice assistants and chatbots. By training a neural network on large amounts of text data, the network can learn to identify patterns and relationships in language and generate responses to queries in a natural and conversational way.
Financial Forecasting: Neural networks can be used to analyze financial data and make predictions about future trends and patterns. This is particularly useful in areas such as stock market analysis and investment management, where accurate predictions can lead to significant financial gains.
Medical Diagnosis: Neural networks can be used to analyze medical data, such as patient records and imaging scans, and make accurate diagnoses. This is particularly useful in areas such as radiology and pathology, where the ability to quickly and accurately identify diseases can save lives.
Autonomous Vehicles: Neural networks are a key technology in the development of autonomous vehicles, such as self-driving cars. By analyzing data from sensors such as cameras and lidar, neural networks can make decisions about how the vehicle should navigate its environment and avoid obstacles.

Conclusion

In conclusion, neural networks have a wide range of applications across many different industries. As more data becomes available, and as neural network technology continues to improve, we can expect to see even more innovative applications of this powerful technology in the future. If you are interested in learning more about neural networks and their applications, there are many online courses and resources available to help you get started.

The neural network is a complex subject to grasp. So far, we only discussed feed-forward neural networks where you have inputs that enter the network and multiply them with weights as they propagate through the layers.

But there are more types of neural networks that you may need to understand, for example, convolutional neural network (CNN), recurrent neural network (RNN), etc.

Here is a chart on the types of neural networks from the Asimov Institute:

I plan to continue this neural network and deep learning series to talk more about CNN & RNN. But for now, let’s keep our explanation simple if you are a complete beginner.

If you are interested to learn more about neural network and deep learning, check out the following tutorials from Linkedin Learning:

What do you think about the general idea behind computers trying to imitate the human brain? Do you believe perceptrons can ever be accurate as biological neurons?