Elizabeth Hill

Brian Gaines

March 27, 2018

Assignment 4

What the Heck is a Neural Network?

What the heck is a neural network? To answer this question, we must ask a couple

more. For starters, what is a network? What does neural mean? How in the world do these two

concepts come together? And finally, what sort of things can neural networks do? Each of these

are loaded questions and will need to be broken down into smaller ideas in order for them to

be easier to swallow. That is what abstraction is for! Abstraction is the amazing process of

taking difficult concepts and presenting them in ways that are relatable and easy to

understand. Most abstractions make hard concepts easy to interpret because they utilize the

pre-existing working knowledge of the person who is trying to learn more about the topic at

hand. By taking the big idea of the neural network and breaking it down into smaller ideas,

adding relatable abstractions, and giving real-world examples of neural networks in action, we

can figure out what the heck a neural network is.

 What is a network? In everyday uses, a network is like a connected group of people or

things that work together. We all have networks in our lives. We have a network of humans

that we know. Let’s call this People. People has different branches: Friends, Enemies,

Classmates, Family, etc. and each of those branches is its own network. We are going to take

Family and look at it more in-depth. For

example, Family can have: mom, dad,

brother, grandma, Uncle Bob (most

people have this one), cousin It, etc. The

network Family can grow and shrink for

many reasons: people get married,

divorced, have kids, exile other family

members and so on. But it doesn’t stop

there, the depth can recursively continue,

Figure 1:Data Visualization for Human Perception. (The Interaction
Design Foundation)
because if you take any member of

Family, such as mom, you can see that she has her own network of People. Her People also has

different branches. They may be similar, but they may be different. Her People could consist of:

Family, Friends, Enemies, Work Related, Tennis Team, Garden Club, etc. Each of these networks

I have mentioned can have anywhere from zero to seven billion people in them. Which brings

the next point, what if someone is in Enemy, but also in Classmates? That is great! This means

that this one person who is lucky enough to be in two of your networks has now connected

these networks. However, usually this one-point-connection is not how networks work, usually

they are quite interconnected! You can see this by analyzing your own branches of Person and
quickly realize that there are many overlapping people throughout your networks. People is

visualized nicely in the image included above, with you at the center.

When talking about a network in computer science, the same principles apply. A

computer network is just like the Person network, except instead of people, there are nodes.

Nodes are points of contact for data being sent between hosts and “just a place where

computation happens”(DL4J). This means that throughout an entire computer network, there

are tons of pieces of information available for accessing and sharing. Each of these pieces of

data can be used in combination with each other in order to follow algorithms and compute

mathematical equations.

Switching gears, we now must figure out what the word neural means. Starting with the

definition, neural means “of, relating to, or affecting a nerve or the nervous system” (Merriam-

Webster). The part of the definition that we want to focus on here is the nervous system. The

nervous system in the human body is what contains all of the different nerves, or neurons, in

your body. Neurons are taking in and sending out electrical signals that carry data to other

neurons. We have these neurons spread out all throughout our bodies, which means that our

nervous system is actually a neural network!

As you could probably guess, this neural network is exactly what our computer

(artificial) neural network is modelled after. The reason why the abstract neural network is

based on the nervous system is in the way it functions. Think of each of the neurons in our

body as a node in a computer network. “Just like a biological neuron has dendrites to receive

signals, a cell body to process them, and an axon to send signals out to other neurons, the
 artificial neuron has a number of input channels, a

processing stage, and one output that can fan out

to multiple other artificial neurons.” (Applied Go).

By now, the connections about networks and

neurons should be firing. It is interesting to note

Figure 2: Biological and artificial neurons (Applied Go)
that this entire paper is about how explaining

things through abstraction is the best way to understand super-complex ideas, so it is

important to point out that the “artificial neural network” got its name because it is an

abstraction of the nervous system. Meta.

Looking deeper into the specifics of the interworking of the neuron, more commonly

called a perceptron when talking about artificial neural networks, we can really see why this

system is so powerful. Neural networks are data in/data out, meaning that there is a bunch of

raw data somewhere that a user has and wants to use as an input in a neural net for whatever

their final goal is. After they put all of this data into the system, they will be getting data back

out that takes into account

and runs calculations on

all of data that was put in

(later, I will walk through a

specific example to

Figure 3: Neuron and Neural Network (Knoldus) demonstrate). First, in

order to visualize an abstract neural network, think of it as being built up of layers of

perceptrons, each layer being one step closer to the final results than the last. The whole
neural network begins at the first layer. Each number in this raw data set will be input into a

perceptron in the first layer, where specific calculations will be done in order to get a new piece

of data that will be sent on to the second layer. But which perceptrons in the second layer will

the first layer outputs go to? Each of the perceptrons in the second layer has a range that they

want to take in as their input. If the number a perceptron from the first layer outputs falls into

this range, it will be sent to that specific second perceptron. It’s kind of like snail mail. The first

output is like a letter to be sent to a mailbox in the second layer. If the address on the letter

corresponds to an address of a mailbox, the letter goes in, and the mailbox is in use; this is

called activation. Then the same thing happens again at the second layer, where an activated

perceptron takes many outputs of different first layer perceptrons to create a new second-layer

output. This process continues on for as many layers that exist in the neural network, which is a

number that depends on the network. The network can have as little as 2 layers, or as many as

you want, but it is important to note that more does not equal better. The overall added

effectiveness of the system decreases with the addition of new layers.

In order to see how this all works in the real world, we must discuss how machine

learning comes into play with neural networks. Machine learning is a type of Artificial

Intelligence (AI) that uses a neural network as its learning infrastructure. Backing up, AI is

basically the study/theory of trying to make computers think like humans do. Since humans use

brains and the nervous system, it is only natural that artificial neural networks are heavily

implemented in training computers to think. The way that machine learning works is by running

hundreds to hundreds of thousands of sets of training data through a neural network. The

computer learns about the data and processes it through each layer of the network. The
programmer has made the algorithms for each layer to act appropriately with the data.

However, that is not the end. Just like humans, computers don’t always get it right the first

couple (hundred) tries. There is this thing called bias that is added into the calculation of values

to account for the computer’s misinterpretation of data, but that’s all we need to know about

it. It is like if we get an answer wrong on a test and an excellent teacher writes in the margin,

“This isn’t technically right, but you were almost there so I’ll just fix it and give it to you.”

Finally, after getting to the end of all of the layers and all of the calculations, a final result will

be produced! The only way that this could be exciting is if we see it play out in a real-life

example.

The “Hello World” of the machine-learning-using-a-neural-network projects is optical

character recognition. This is when you handwrite a number on a touch screen and the

computer takes the scribble and converts it into an ASCII character (American Standard Code

for Information Interchange, in other words, a digital number). How does

the computer know what that scribble means? Through sending

thousands and thousands of scribbles, like the ones to the left (MNIST),

through the network to train it. Since humans can look at essentially any
Figure 4:Sample
characters(MNIST) handwritten number and know what the value is, we can assign each of

the scribble images a value and teach the computer that that image and that value go together.

This is known as the training stage. Specifically, this is called supervised learning, which is when

the “algorithms are trained using labeled examples, such as an input where the desired output

is known” (SAS). Over and over we are drilling into the computers neurons that there is some

sort of relationship between an image and a value, and we do this so well that eventually we
could give it a new character and it would know what to do all on its own. So, to think about

what is actually going on in the neural network, we need to break this process down into layers

(3Blue1Brown is to thank for this explanation). Only one number is sent through the network at

a time. To make the math easy, if our images of numbers were 20 pixels by 20 pixels, the first

layer of the network would have 400 perceptrons, and each pixel of that image would go to a

perceptron one-to-one. The value of the perceptron would be 0 if the pixel was black, and 1 if

the pixel was white. Things aren’t all black and white though. If the pixel isn’t pure black and

isn’t pure white, that’s okay! This just means that the perceptrons value will be anywhere from

0.00 to 1.00. So, if the pixel is a dark grey, its perceptrons value could be 0.23. Once the first

layer has been assigned, the perceptrons will fire off to their special destinations in the second

layer? How will they know where to go in a way that will help solve what the mystery number

is? By grouping pixels of similar

segments together. We know

that all numbers are pretty

much made up of the same

segments, and this can be seen

through looking at the digital

Figure 5: 9 being sent through the layers of a Neural Network (3B1B)

display on a microwave. “Simple intuitions about how we recognize shapes - "a 9 has a loop at

the top, and a vertical stroke in the bottom right" - turn out to be not so simple to express

algorithmically. Neural networks approach the problem in a different way, [they] use the

[training] examples to automatically infer rules for recognizing handwritten digits” (Nielson).

Groups of pixels that could be part of the same segment all get put into the same perceptron
on the next layer. This means that there are many less perceptrons on the second layer, which

is good because we are getting closer to one answer. Likewise, there is a final third layer that

has only 10 perceptrons: one for each digit 0-9. The segments lit up in the second layer will be

sent to the final layer with the digit that they represent the best mathematically, and finally one

digit in the last layer will be activated the most! The digit with the highest activation is the

computer’s best guess (and it’s usually spot on) at what our scribble is.

Hopefully now, through the help of dividing, abstraction, and example, we can finally

answer the question of what the heck a neural network is.

 Works Cited

3Blue1Brown. But What *Is* a Neural Network? | Chapter 1, Deep Learning. Youtube, 5 Oct. 2017,

But what *is* a Neural Network? | Chapter 1, deep learning.

“A Biological and an Artificial Neuron.” Applied Go, appliedgo.net/media/perceptron/neuron.png.

“Data Visualization for Human Perception.” The Interaction Design Foundation, www.interaction-

design.org/literature/book/the-encyclopedia-of-human-computer-interaction-2nd-ed/data-

visualization-for-human-perception?refresh=true%3Frefresh%3Dtrue.

“Data Visualization for Human Perception.” The Interaction Design Foundation, www.interaction-

design.org/literature/book/the-encyclopedia-of-human-computer-interaction-2nd-ed/data-

visualization-for-human-perception?refresh=true%3Frefresh%3Dtrue.

“First Interaction Artificial Neural Network.” Knoldus, pranjut.files.wordpress.com/2017/06/ann-vs-

bnn.jpg?w=640.

“Machine Learning: What It Is and Why It Matters.” What It Is and Why It Matters | SAS,

www.sas.com/en_us/insights/analytics/machine-learning.html.

“Neural.” Merriam-Webster, Merriam-Webster, www.merriam-webster.com/dictionary/neural.

Nicholson, Chris V., and Adam Gibson. “Introduction to Deep Neural Networks (Deep

Learning).” Introduction to Deep Neural Networks (Deep Learning) - Deeplearning4j: Open-

Source, Distributed Deep Learning for the JVM, deeplearning4j.org/neuralnet-overview.

Nielsen, Michael A. Neural Networks and Deep Learning. Determination Press, 2015.

“Perceptrons - the Most Basic Form of a Neural Network.” Applied Go, 9 June 2016,

appliedgo.net/perceptron/.