You are on page 1of 34

 Ho m e

 Ab o u t
 Ar c h i v e s
 Co n ta c t Us
 Re s o u r c e s

Difference Between Similar Terms and Objects

Search

o Business
 ACCOUNTING
 ECONOMICS
 FINANCE
 INVESTMENT
 MANAGEMENT
 MARKETING
 ORGANIZA TIONS
 PLANNING & A CTIV IT IE S
 PRODUCT & SERVI CES
 STRUCTURE & S YSTEMS
o Miscellaneous
 CAREER & EDUC AT ION
 CULTURE
 ENTERTAINMENT
 FASHION & B EAUT Y
 GEOGRAPHY
 HISTORY
 LEGA L
 POLITIC S
 IDEOLOG Y
 LEADERS
 POLITIC AL INSTI TUTIO NS
 RELIGION
 ISLAM
 SPORTS
o Technolog y
 GAD GETS
 GAMING
 SMARTPHONES
 TABLE TS
 SOFTWARE
 INTERNET
 HARDWARE
 PROTOCOLS & FORMAT S
 COMMUNICATION
 WEB APPLI CA TIONS
 INDUSTRIAL
 HOUSEHOLD EQUIPMENTS
 AUTO
 CAREER AND CERT IFIC ATIONS
o Science
 BIOLOGY
 CHEMISTRY
 HEAL TH
 DISEASE
 DIET & FI TNESS
 DRUGS
 MATHEMAT ICS & ST AT I STICS
 NATURE
 ANIMALS
 BIRDS
 PHYSICS
 PSYCHOLOG Y
o Objects
 FOOD
 PROCESSED FOODS
 VEGET ABLES & FRUIT S
o Language
 WORDS
 GRAMMAR

Differences Between Supervised Learning and Unsupervised


Learning
• Categorized under Protocols & Formats,Software,Technology,Web Applications | Differences Between Supervised
Learning and Unsupervised Learning
Students venturing in machine learning have been experiencing difficulties in differentiating supervised learning
from unsupervised learning. It appears that the procedure used in both learning methods is the same, which makes it
difficult for one to differentiate between the two methods of learning. However, upon scrutiny and unwavering
attention, one can clearly understand that there exist significant differences between supervised and unsupervised
learning.
 What is Supervised Learning?
Supervised learning is one of the methods associated with machine learning which involves allocating labeled data so
that a certain pattern or function can be deduced from that data. It is worth noting that supervised learning involves
allocating an input object, a vector, while at the same time anticipating the most desired output value, which is mostly
referred to as the supervisory signal. The bottom line property of supervised learning is that the input data is known
and labeled appropriately.

 What is Unsupervised Learning?


Unsupervised learning is the second method of machine learning algorithm where inferences are drawn from
unlabeled input data. The goal of unsupervised learning is to determine the hidden patterns or grouping in data from
unlabeled data. It is mostly used in exploratory data analysis. One of the defining characters of unsupervised learning
is that both the input and output are not known.

Differences Between Supervised Learning and Unsupervised Learning


1. Input Data in Supervised Learning and Unsupervised Learning

The primary difference between supervised learning and unsupervised learning is the data used in either method of
machine learning. It is worth noting that both methods of machine learning require data, which they will analyze to
produce certain functions or data groups. However, the input data used in supervised learning is well known and is
labeled. This means that the machine is only tasked with the role of determining the hidden patterns from already
labeled data. However, the data used in unsupervised learning is not known nor labeled. It is the work of the machine
to categorize and label the raw data before determining the hidden patterns and functions of the input data.
Report this ad

2. Computational Complexity in Supervised Learning and Unsupervised Learning

Machine learning is a complex affair and any person involved must be prepared for the task ahead. One of the stand
out differences between supervised learning and unsupervised learning is computational complexity. Supervised
learning is said to be a complex method of learning while unsupervised method of learning is less complex. One of the
reason that makes supervised learning affair is the fact that one has to understand and label the inputs while in
unsupervised learning, one is not required to understand and label the inputs. This explains why many people have
been preferring unsupervised learning as compared to the supervised method of machine learning.

3. Accuracy of the Results of Supervised Learning and Unsupervised Learning

The other prevailing difference between supervised learning and unsupervised learning is the accuracy of the results
produced after every cycle of machine analysis. All the results generated from supervised method of machine learning
are more accurate and reliable as compared to the results generated from the unsupervised method of machine
learning. One of the factor that explains why supervised method of machine learning produces accurate and reliable
results is because the input data is well known and labeled which means that the machine will only analyze the hidden
patterns. This is unlike unsupervised method of learning where the machine has to define and label the input data
before determining the hidden patterns and functions.

4. Number of Classes in Supervised Learning and Unsupervised Learning

It is also worth noting that there is a significant difference when it comes to the number of classes. It is worth noting
that all the classes used in supervised learning are known which means that also the answers in the analysis are likely
to be known. The only goal of supervised learning is therefore to determine the unknown cluster. However, there is no
prior knowledge in unsupervised method of machine learning. In addition, the numbers of classes are not known
which clearly means that no information is known and the results generated after the analysis cannot be ascertained.
Moreover, the people involved in unsupervised method of learning are not aware of any information concerning the
raw data and the expected results.

5. Real Time Learning in Supervised Learning and Unsupervised Learning

Among other differences, there exist the time after which each method of learning takes place. It is important to
highlight that supervised method of learning takes place off-line while unsupervised method of learning takes place in
real time. People involved in preparation and labeling of the input data do so off-line while the analysis of the hidden
pattern is done online which denies the people involved in machine learning an opportunity to interact with the
machine as it analyzes the discrete data. However, unsupervised method of machine learning takes place in real time
such that all the input data is analyzed and labeled in the presence of learners which helps them to understand
different methods of learning and classification of raw data. Real time data analysis remains to be the most significant
merit of unsupervised method of learning.

TABLE SHOWING DIFFERENCES BETWEEN SUPERVISED LEARNING AND UNSUPERVISED


LEARNING: COMPARISON CHART

Supervised Learning Unsupervised Learning

Uses Known and Labeled Uses Unknown Input


Input Data
Input Data Data

Computational Very Complex in Less Computational


Complexity Computation Complexity

Uses Real Time Analysis


Real Time Uses off-line analysis
of Data

Number of Classes is Number of Classes is not


Number of Classes
Known Known

Accurate and Reliable Moderate Accurate and


Accuracy of Results
Results Reliable Results

Summary of Supervised Learning and Unsupervised Learning


 Data mining is becoming an essential aspect in the current business world due to increased raw data that
organizations need to analyze and process so that they can make sound and reliable decisions.
 This explains why the need for machine learning is growing and thus requiring people with sufficient knowledge
of both supervised machine learning and unsupervised machine learning.
 It is worth understanding that each method of learning offers its own advantages and disadvantages. This means
that one has to be conversant with both methods of machine learning before determine which method one will
use to analyze data.
 About
 Latest Posts

Jecinta Morgan

Facebook3Pinterest 99 TwitterPrintShare

Search DifferenceBetween.net :

Search

Custom Search

Help us improve. Rate this post!

Email This Post : If you like this article or our site. Please spread the word. Share it with your friends/family.

 Difference between Clustering and Classification

 Differences Between Machine Learning and Deep Learning

 Difference Between Data Hiding and Data Encapsulation

 Difference between Qualitative Analysis and Quantitative Analysis

 Difference Between Python and R Machine Learning

 Difference Between NFA and DFA

 Difference Between Inductive and Deductive Language Teaching and Learning

Cite
Jecinta Morgan. "Differences Between Supervised Learning and Unsupervised Learning." DifferenceBetween.net.
June 21, 2018 < http://www.differencebetween.net/technology/differences-between-supervised-learning-and-
unsupervised-learning/ >.

Leave a Respon se

Name ( required )
Email ( required )

Website

Please note: comment moderation is enabled and may delay your comment. There is no need to resubmit your
comment.

Submit Comment

Notify me of followup comments via e-mail

Written by : Jecinta Morgan. and updated on June 21, 2018

References :
[0]Carpenter, Gail A., Stephen Grossberg, and John H. Reynolds. "ARTMAP: Supervised real-time learning and
classification of nonstationary data by a self-organizing neural network." Neural networks 4.5 (1991): 565-588.

[1]Fritzke, Bernd. "Growing cell structures—a self-organizing network for unsupervised and supervised learning."
Neural networks 7.9 (1994): 1441-1460.

[2]Hastie, Trevor, Robert Tibshirani, and Jerome Friedman. "Overview of supervised learning." The elements of
statistical learning. Springer New York, 2009. 9-41.

[3]"Image Credit: https://nyghtowl.io/category/data-science/machine-learning/"

Articles on DifferenceBetween.net are general information, and are not intended to substitute for professional advice.
The information is "AS IS", "WITH ALL FAULTS". User assumes all risk of use, damage, or injury. You agree that we
have no liability for any damages.

See more about : Supervised Learning, Unsupervised Learning

Get New Comparisons in your inbox:

Email
Subscribe

More in 'Protocols & F ormats'

 Difference Between Compiled and Interpreted Language


 Difference Between Nexus and Global Entry Program
 Difference between JSON and XML
 Difference between JPG and JPEG

More in 'Software'

 Difference Between IBM and SCO


 Difference Between IBM and Accenture
 Difference Between Adobe Muse and Dreamweaver
 Difference Between Scripting Language and Programming Language

More in 'Technology'

 Difference Between IBM and SCO


 Difference Between VRBO and HomeAway
 Difference Between Lithograph and Print
 Difference Between IBM and Accenture

More in 'Web Ap plications'

 Difference Between Adobe Muse and Dreamweaver


 Difference Between OOP and Functional Programming
 Difference Between Norton and Malwarebytes
 Difference Between McAfee and Webroot

Follow Us

FacebookTwitterPinterestFeed
Most Emai led Comparisons

 Difference Between Vitamin D and Vitamin D3 - 118 emails


 Difference Between Goals and Objectives - 102 emails
 Difference Between LCD and LED Televisions - 89 emails
 Difference Between Islam and Muslim - 87 emails
 Difference Between Objective and Subjective - 66 emails
 Difference Between Oxycontin and Oxycodone - 65 emails

Editor's Picks

 Difference Between Gen Z and Millennials


 Difference Between Mark Zuckerberg and Bill Gates
 Difference Between Civil War and Revolution
 Difference Between Aperol and Campari
 Difference Between Queen and Full
Copyright © 2019 Difference Between • Use of this web site constitutes acceptance of the Terms of Use and Privacy Policy stated
here :
 Legal

Exclusive Member of Mediavine Home

Subscribe • Comments

Read more: Differences Between Supervised Learning and Unsupervised Learning | Difference
Between http://www.differencebetween.net/technology/differences-between-supervised-learning-
and-unsupervised-learning/#ixzz5pC5X8Aht
Skip to content

The Official NVIDIA Blog

USA - United States

 PLATFORMS
o AI AND DEEP LEARNING
 INDUSTRIES
 OVERVIEW
 AI INNOVATORS
 AI CITIES
 AI FOR PUBLIC GOOD
 HEALTHCARE
 HIGHER EDUCATION
 RETAIL
 ROBOTICS
 SELF DRIVING CARS
 DEVELOPER
 PRODUCTS
 AGX SYSTEMS
 DGX SYSTEMS
 NVIDIA GPU CLOUD
 NVIDIA TITAN V
 SOLUTIONS
 INFERENCE
 EDUCATION
 AI STARTUPS
o CUDA
o DATA CENTER
 PRODUCTS
 TESLA
 DGX
 DGX-2
 DGX STATION
 HGX
 NVIDIA GPU CLOUD
 SOLUTIONS
 AI AND DEEP LEARNING
 HIGH PERFORMANCE COMPUTING
 GPU CLOUD COMPUTING
 GPU VIRTUALIZATION
 ANALYTICS
 APPS
 GPU APPS DIRECTORY
 GPU READY APPS
 FOR DEVELOPERS
 TECHNOLOGIES
 NVIDIA VOLTA
 NVIDIA PASCAL
 NVLINK/NVSWITCH
 TENSOR CORES
 INDEX PARAVIEW PLUGIN
o DESIGN AND PRO VISUALIZATION
 INDUSTRIES
 ARCHITECTURE, ENGINEERING AND CONSTRUCTION
 EDUCATION
 MANUFACTURING
 MEDIA AND ENTERTAINMENT
 PRODUCTS
 QUADRO
 QUADRO vDWS
 GRID vPC/vAPPS
 NVIDIA TITAN Xp
 NVS
 SOLUTIONS
 MULTI-DISPLAY
 RENDERING
 VIRTUALIZATION
 VIRTUAL REALITY
 HPC VISUALIZATION
 TECHNOLOGIES
 MATERIAL DEFINITION LANGUAGE
 NVIDIA RTX
 NVLINK
 TURING ARCHITECTURE
 VIRTUAL GPU TECHNOLOGY
 HOLODECK
 FOR DEVELOPERS
o AUTONOMOUS MACHINES
 INDUSTRIES
 DRONES
 ROBOTICS
 INTELLIGENT VIDEO ANALYTICS (IVA)
 PRODUCTS
 FOR DEVELOPERS
o SELF DRIVING CARS
 SOLUTIONS
 NVIDIA DRIVE PLATFORM
 NVIDIA DGX SYSTEMS
 NVIDIA DRIVE CONSTELLATION
 NVIDIA DRIVE IX
 HD MAPPING
 ADVANCED DRIVER ASSISTANCE SYSTEMS
 PARTNERS
 FOR DEVELOPERS
o GEFORCE GAMING
 20-SERIES GRAPHICS CARDS
 10-SERIES GRAPHICS CARDS
 LAPTOPS
 G-SYNC MONITORS
 GEFORCE EXPERIENCE
 GEFORCE NOW
o SHIELD
 SHIELD TV
 BIG FORMAT GAMING DISPLAYS (BFGD)
 AI FOR THE HOME
 DEVELOPERS
o DEVELOPER PROGRAM
o CUDA
o TRAINING
o GPU TECH CONFERENCE
 COMMUNITY
o NVIDIA BLOG
o GEFORCE FORUMS
o VIRTUAL GPU FORUMS
o GPU VENTURES
o INCEPTION PROGRAM
 SHOP
 DRIVERS
o GEFORCE DRIVERS
o ALL NVIDIA DRIVERES
 SUPPORT
 ABOUT NVIDIA
o AI COMPUTING MODEL
o NEWSROOM
o NVIDIA BLOG
o RESEARCH
o WEBINARS
o EVENTS
o COMPANY INFORMATION
o CAREERS
o INVESTORS
o SOCIAL RESPONSIBILITY

 HOME

 DEEP LEARNING

 DRIVING

 GAMING

 PRO GRAPHICS

 AUTONOMOUS MACHINES

 HEALTHCARE
 AI PODCAST

SEARCH FOR:

Search

SuperVize Me: What’s the Difference Between


Supervised, Unsupervised, Semi-Supervised and
Reinforcement Learning?
August 2, 2018 by ISHA SALIAN

321 Shares




There are a few different ways to build IKEA furniture. Each will, ideally, lead to a completed couch or chair. But depending on the details, one
approach will make more sense than the others.

Got the instruction manual and all the right pieces? Just follow directions. Getting the hang of it? Toss the manual aside and go solo. But
misplace the instructions, and it’s up to you to make sense of that pile of wooden dowels and planks.

It’s the same with deep learning. Based on the kind of data available and the research question at hand, a scientist will choose to train an
algorithm using a specific learning model.

In a supervised learning model, the algorithm learns on a labeled dataset, providing an answer key that the algorithm can use to evaluate its
accuracy on training data. An unsupervised model, in contrast, provides unlabeled data that the algorithm tries to make sense of by extracting
features and patterns on its own.

Semi-supervised learning takes a middle ground. It uses a small amount of labeled data bolstering a larger set of unlabeled data. And
reinforcement learning trains an algorithm with a reward system, providing feedback when an artificial intelligence agent performs the best action
in a particular situation.

Let’s walk through the kinds of datasets and problems that lend themselves to each kind of learning.

What Is Supervised Learning?


If you’re learning a task under supervision, someone is present judging whether you’re getting the right answer. Similarly, in supervised learning,
that means having a full set of labeled data while training an algorithm.

Fully labeled means that each example in the training dataset is tagged with the answer the algorithm should come up with on its own. So, a
labeled dataset of flower images would tell the model which photos were of roses, daisies and daffodils. When shown a new image, the model
compares it to the training examples to predict the correct label.
With supervised machine learning, the algorithm learns from labeled data.
There are two main areas where supervised learning is useful: classification problems and regression problems.

Cat, koala or turtle? A classification algorithm can


tell the difference.
Classification problems ask the algorithm to predict a discrete value, identifying the input data as the member of a particular class, or group. In a
training dataset of animal images, that would mean each photo was pre-labeled as cat, koala or turtle. The algorithm is then evaluated by how
accurately it can correctly classify new images of other koalas and turtles.

On the other hand, regression problems look at continuous data. One use case, linear regression, should sound familiar from algebra class: given a
particular x value, what’s the expected value of the y variable?

A more realistic machine learning example is one involving lots of variables, like an algorithm that predicts the price of an apartment in San
Francisco based on square footage, location and proximity to public transport.

Supervised learning is, thus, best suited to problems where there is a set of available reference points or a ground truth with which to train the
algorithm. But those aren’t always available.
What Is Unsupervised Learning?
Clean, perfectly labeled datasets aren’t easy to come by. And sometimes, researchers are asking the algorithm questions they don’t know the
answer to. That’s where unsupervised learning comes in.

In unsupervised learning, a deep learning model is handed a dataset without explicit instructions on what to do with it. The training dataset is a
collection of examples without a specific desired outcome or correct answer. The neural network then attempts to automatically find structure in
the data by extracting useful features and analyzing its structure.

Unsupervised learning models automatically extract features and find patterns in the data.
Depending on the problem at hand, the unsupervised learning model can organize the data in different ways.

 Clustering: Without being an expert ornithologist, it’s possible to look at a collection of bird photos and separate them roughly by species,
relying on cues like feather color, size or beak shape. That’s how the most common application for unsupervised learning, clustering, works: the
deep learning model looks for training data that are similar to each other and groups them together.
 Anomaly detection: Banks detect fraudulent transactions by looking for unusual patterns in customer’s purchasing behavior. For instance, if the
same credit card is used in California and Denmark within the same day, that’s cause for suspicion. Similarly, unsupervised learning can be used
to flag outliers in a dataset.
 Association: Fill an online shopping cart with diapers, applesauce and sippy cups and the site just may recommend that you add a bib and a baby
monitor to your order. This is an example of association, where certain features of a data sample correlate with other features. By looking at a
couple key attributes of a data point, an unsupervised learning model can predict the other attributes with which they’re commonly associated.
 Autoencoders: Autoencoders take input data, compress it into a code, then try to recreate the input data from that summarized code. It’s like
starting with Moby Dick, creating a SparkNotes version and then trying to rewrite the original story using only SparkNotes for reference. While a
neat deep learning trick, there are fewer real-world cases where a simple autocoder is useful. But add a layer of complexity and the possibilities
multiply: by using both noisy and clean versions of an image during training, autoencoders can remove noise from visual data like images, video
or medical scans to improve picture quality.

Because there is no “ground truth” element to the data, it’s difficult to measure the accuracy of an algorithm trained with unsupervised learning.
But there are many research areas where labeled data is elusive, or too expensive, to get. In these cases, giving the deep learning model free rein
to find patterns of its own can produce high-quality results.

What Is Semi-Supervised Learning?


Think of it as a happy medium.

Semi-supervised learning is, for the most part, just what it sounds like: a training dataset with both labeled and unlabeled data. This method is
particularly useful when extracting relevant features from the data is difficult, and labeling examples is a time-intensive task for experts.
Semi-supervised learning is especially useful for
medical images, where a small amount of labeled data can lead to a significant improvement in accuracy.
Common situations for this kind of learning are medical images like CT scans or MRIs. A trained radiologist can go through and label a small
subset of scans for tumors or diseases. It would be too time-intensive and costly to manually label all the scans — but the deep learning network
can still benefit from the small proportion of labeled data and improve its accuracy compared to a fully unsupervised model.

A popular training method that starts with a fairly small set of labeled data is using general adversarial networks, or GANs.

Imagine two deep learning networks in competition, each trying to outsmart the other. That’s a GAN. One of the networks, called the generator,
tries to create new data points that mimic the training data. The other network, the discriminator, pulls in these newly generated data and
evaluates whether they are part of the training data or fakes. The networks improve in a positive feedback loop — as the discriminator gets better
at separating the fakes from the originals, the generator improves its ability to create convincing fakes.

This is how a GAN works: The discriminator, labeled “D,” is shown images from both the generator, “G,” and from the training dataset. The
discriminator is tasked with determining which images are real, and which are fakes from the generator.
What Is Reinforcement Learning?
Video games are full of reinforcement cues. Complete a level and earn a badge. Defeat the bad guy in a certain number of moves and earn a
bonus. Step into a trap — game over.

These cues help players learn how to improve their performance for the next game. Without this feedback, they would just take random actions
around a game environment in the hopes of advancing to the next level.

Reinforcement learning operates on the same principle — and actually, video games are a common test environment for this kind of research.

In this kind of machine learning, AI agents are attempting to find the optimal way to accomplish a particular goal, or improve performance on a
specific task. As the agent takes action that goes toward the goal, it receives a reward. The overall aim: predict the best next step to take to earn
the biggest final reward.

To make its choices, the agent relies both on learnings from past feedback and exploration of new tactics that may present a larger payoff. This
involves a long-term strategy — just as the best immediate move in a chess game may not help you win in the long run, the agent tries to
maximize the cumulative reward.

It’s an iterative process: the more rounds of feedback, the better the agent’s strategy becomes. This technique is especially useful for training
robots, which make a series of decisions in tasks like steering an autonomous vehicle or managing inventory in a warehouse.

Just as students in a school, every algorithm learns differently. But with the diversity of approaches available, it’s only a matter of picking the
best way to help your neural network learn the ropes.

Categories: Deep Learning | Explainer


Tags: Artificial Intelligence | Machine Learning

LISTEN IN TO THE AI PODCAST


LISTEN NOW

ALL NVIDIA NEWS


AI Potcast: A Joint Discussion on AI, Agtech with Grownetics CEO
May 27, 2019
An On-Ramp for AI: NVIDIA Expands NGC-Ready Portfolio and Support Services
May 27, 2019
NVIDIA Kicks Off Computex, Supercharging Computers for Gamers and Creators, Driving AI at the Edge
May 27, 2019
World’s Most Powerful Mobile Workstations Launch with NVIDIA Quadro RTX, Delivering AI, Ray Tracing, VR
May 27, 2019
GPU Computing 101: Why University Educators Are Pulling NVIDIA Teaching Kits into Their Classrooms
May 24, 2019

Post navigation

What’s the Difference Between Artificial


Intelligence, Machine Learning, and Deep
Learning?
July 29, 2016 by MICHAEL COPELAND

5003 Shares




This is the first of a multi-part series explaining the fundamentals of deep learning by long-time tech journalist Michael Copeland.

Artificial intelligence is the future. Artificial intelligence is science fiction. Artificial intelligence is already part of our everyday lives. All those
statements are true, it just depends on what flavor of AI you are referring to.

For example, when Google DeepMind’s AlphaGo program defeated South Korean Master Lee Se-dol in the board game Go earlier this year, the
terms AI, machine learning, and deep learning were used in the media to describe how DeepMind won. And all three are part of the reason why
AlphaGo trounced Lee Se-Dol. But they are not the same things.

The easiest way to think of their relationship is to visualize them as concentric circles with AI — the idea that came first — the largest, then
machine learning — which blossomed later, and finally deep learning — which is driving today’s AI explosion — fitting inside both.
From Bust to Boom
AI has been part of our imaginations and simmering in research labs since a handful of computer scientists rallied around the term at the
Dartmouth Conferences in 1956 and birthed the field of AI. In the decades since, AI has alternately been heralded as the key to our civilization’s
brightest future, and tossed on technology’s trash heap as a harebrained notion of over-reaching propellerheads. Frankly, until 2012, it was a bit
of both.

Over the past few years AI has exploded, and especially since 2015. Much of that has to do with the wide availability of GPUs that make parallel
processing ever faster, cheaper, and more powerful. It also has to do with the simultaneous one-two punch of practically infinite storage and a
flood of data of every stripe (that whole Big Data movement) – images, text, transactions, mapping data, you name it.
Let’s walk through how computer scientists have moved from something of a bust — until 2012 — to a boom that has unleashed applications
used by hundreds of millions of people every day.

Artificial Intelligence — Human Intelligence Exhibited by Machines

King me: computer programs that played checkers were among the earliest examples of artificial intelligence, stirring an early wave of
excitement in the 1950s.
Back in that summer of ’56 conference the dream of those AI pioneers was to construct complex machines — enabled by emerging computers —
that possessed the same characteristics of human intelligence. This is the concept we think of as “General AI” — fabulous machines that have all
our senses (maybe even more), all our reason, and think just like we do. You’ve seen these machines endlessly in movies as friend — C-3PO —
and foe — The Terminator. General AI machines have remained in the movies and science fiction novels for good reason; we can’t pull it off, at
least not yet.

What we can do falls into the concept of “Narrow AI.” Technologies that are able to perform specific tasks as well as, or better than, we humans
can. Examples of narrow AI are things such as image classification on a service like Pinterest and face recognition on Facebook.

Those are examples of Narrow AI in practice. These technologies exhibit some facets of human intelligence. But how? Where does that
intelligence come from? That get us to the next circle, machine learning.

Machine Learning — An Approach to Achieve Artificial Intelligence

Spam free diet: machine learning helps keep your inbox (relatively) free of spam.
Machine learning at its most basic is the practice of using algorithms to parse data, learn from it, and then make a determination or prediction
about something in the world. So rather than hand-coding software routines with a specific set of instructions to accomplish a particular task, the
machine is “trained” using large amounts of data and algorithms that give it the ability to learn how to perform the task.
Machine learning came directly from minds of the early AI crowd, and the algorithmic approaches over the years included decision tree learning,
inductive logic programming. clustering, reinforcement learning, and Bayesian networks among others. As we know, none achieved the ultimate
goal of General AI, and even Narrow AI was mostly out of reach with early machine learning approaches.

To learn more about deep learning, listen to our Deep Learning 101 podcast with NVIDIA’s own Will Ramey.

As it turned out, one of the very best application areas for machine learning for many years was computer vision, though it still required a great
deal of hand-coding to get the job done. People would go in and write hand-coded classifiers like edge detection filters so the program could
identify where an object started and stopped; shape detection to determine if it had eight sides; a classifier to recognize the letters “S-T-O-P.”
From all those hand-coded classifiers they would develop algorithms to make sense of the image and “learn” to determine whether it was a stop
sign.

Good, but not mind-bendingly great. Especially on a foggy day when the sign isn’t perfectly visible, or a tree obscures part of it. There’s a reason
computer vision and image detection didn’t come close to rivaling humans until very recently, it was too brittle and too prone to error.

Time, and the right learning algorithms made all the difference.

Deep Learning — A Technique for Implementing Machine Learning


Herding cats: Picking images of cats out of YouTube videos was one of the first breakthrough demonstrations of deep learning.
Another algorithmic approach from the early machine-learning crowd, artificial neural networks, came and mostly went over the decades. Neural
networks are inspired by our understanding of the biology of our brains – all those interconnections between the neurons. But, unlike a biological
brain where any neuron can connect to any other neuron within a certain physical distance, these artificial neural networks have discrete layers,
connections, and directions of data propagation.

You might, for example, take an image, chop it up into a bunch of tiles that are inputted into the first layer of the neural network. In the first layer
individual neurons, then passes the data to a second layer. The second layer of neurons does its task, and so on, until the final layer and the final
output is produced.
Each neuron assigns a weighting to its input — how correct or incorrect it is relative to the task being performed. The final output is then
determined by the total of those weightings. So think of our stop sign example. Attributes of a stop sign image are chopped up and “examined” by
the neurons — its octogonal shape, its fire-engine red color, its distinctive letters, its traffic-sign size, and its motion or lack thereof. The neural
network’s task is to conclude whether this is a stop sign or not. It comes up with a “probability vector,” really a highly educated guess, based on
the weighting. In our example the system might be 86% confident the image is a stop sign, 7% confident it’s a speed limit sign, and 5% it’s a kite
stuck in a tree ,and so on — and the network architecture then tells the neural network whether it is right or not.

Even this example is getting ahead of itself, because until recently neural networks were all but shunned by the AI research community. They had
been around since the earliest days of AI, and had produced very little in the way of “intelligence.” The problem was even the most basic neural
networks were very computationally intensive, it just wasn’t a practical approach. Still, a small heretical research group led by Geoffrey Hinton at
the University of Toronto kept at it, finally parallelizing the algorithms for supercomputers to run and proving the concept, but it wasn’t
until GPUs were deployed in the effort that the promise was realized.

If we go back again to our stop sign example, chances are very good that as the network is getting tuned or “trained” it’s coming up with wrong
answers — a lot. What it needs is training. It needs to see hundreds of thousands, even millions of images, until the weightings of the neuron
inputs are tuned so precisely that it gets the answer right practically every time — fog or no fog, sun or rain. It’s at that point that the neural
network has taught itself what a stop sign looks like; or your mother’s face in the case of Facebook; or a cat, which is what Andrew Ng did in
2012 at Google.

Ng’s breakthrough was to take these neural networks, and essentially make them huge, increase the layers and the neurons, and then run massive
amounts of data through the system to train it. In Ng’s case it was images from 10 million YouTube videos. Ng put the “deep” in deep learning,
which describes all the layers in these neural networks.

Today, image recognition by machines trained via deep learning in some scenarios is better than humans, and that ranges from cats to identifying
indicators for cancer in blood and tumors in MRI scans. Google’s AlphaGo learned the game, and trained for its Go match — it tuned its neural
network — by playing against itself over and over and over.

Thanks to Deep Learning, AI Has a Bright Future


Deep learning has enabled many practical applications of machine learning and by extension the overall field of AI. Deep learning breaks down
tasks in ways that makes all kinds of machine assists seem possible, even likely. Driverless cars, better preventive healthcare, even better movie
recommendations, are all here today or on the horizon. AI is the present and the future. With Deep learning’s help, AI may even get to that
science fiction state we’ve so long imagined. You have a C-3PO, I’ll take it. You can keep your Terminator.

Categories: Data Center | Deep Learning | Driving | Explainer | Supercomputing | The AI Podcast
Tags: Artificial Intelligence | Data Science | GPU | Machine Learning
LOAD COMMENTS

LISTEN IN TO THE AI PODCAST


LISTEN NOW
ALL NVIDIA NEWS
AI Potcast: A Joint Discussion on AI, Agtech with Grownetics CEO
May 27, 2019
An On-Ramp for AI: NVIDIA Expands NGC-Ready Portfolio and Support Services
May 27, 2019
NVIDIA Kicks Off Computex, Supercharging Computers for Gamers and Creators, Driving AI at the Edge
May 27, 2019
World’s Most Powerful Mobile Workstations Launch with NVIDIA Quadro RTX, Delivering AI, Ray Tracing, VR
May 27, 2019
GPU Computing 101: Why University Educators Are Pulling NVIDIA Teaching Kits into Their Classrooms
May 24, 2019

Post navigation

What’s the Difference Between Deep Learning


Training and Inference?
August 22, 2016 by MICHAEL COPELAND

264 Shares




This is the second of a multi-part series explaining the fundamentals of deep learning by long-time tech journalist Michael Copeland.

School’s in session. That’s how to think about deep neural networks going through the “training” phase. Neural networks get an education for the
same reason most people do — to learn to do a job.

More specifically, the trained neural network is put to work out in the digital world using what it has learned — to recognize images, spoken
words, a blood disease, or suggest the shoes someone is likely to buy next, you name it — in the streamlined form of an application. This speedier
and more efficient version of a neural network infers things about new data it’s presented with based on its training. In the AI lexicon this is
known as “inference.”
Inference is where capabilities learned during deep learning training are put to work.
Inference can’t happen without training. Makes sense. That’s how we gain and use our own knowledge for the most part. And just as we don’t
haul around all our teachers, a few overloaded bookshelves and a red-brick schoolhouse to read a Shakespeare sonnet, inference doesn’t require
all the infrastructure of its training regimen to do its job well.

So let’s break down the progression from training to inference, and in the context of AI how they both function.

Training Deep Neural Networks


Just as we don’t haul around all our teachers, a few overloaded bookshelves and a red-brick schoolhouse to read a Shakespeare sonnet, inference
doesn’t require all the infrastructure of its training regimen to do its job well.
While the goal is the same – knowledge — the educational process, or training, of a neural network is (thankfully) not quite like our own. Neural
networks are loosely modeled on the biology of our brains — all those interconnections between the neurons. Unlike our brains, where any
neuron can connect to any other neuron within a certain physical distance, artificial neural networks have separate layers, connections, and
directions of data propagation.

When training a neural network, training data is put into the first layer of the network, and individual neurons assign a weighting to the input —
how correct or incorrect it is — based on the task being performed.
To learn more, check out NVIDIA’s inference solutions for the data center, self-driving cars, video analytics and more.

In an image recognition network, the first layer might look for edges. The next might look for how these edges form shapes — rectangles or
circles. The third might look for particular features — such as shiny eyes and button noses. Each layer passes the image to the next, until the final
layer and the final output determined by the total of all those weightings is produced.

But here’s where the training differs from our own. Let’s say the task was to identify images of cats. The neural network gets all these training
images, does its weightings and comes to a conclusion of cat or not. What it gets in response from the training algorithm is only “right” or
“wrong.”

Training Is Compute Intensive


And if the algorithm informs the neural network that it was wrong, it doesn’t get informed what the right answer is. The error is propagated back
through the network’s layers and it has to guess at something else. In each attempt it must consider other attributes — in our example attributes of
“catness” — and weigh the attributes examined at each layer higher or lower. Then it guesses again. And again. And again. Until it has the
correct weightings and gets the correct answer practically every time. It’s a cat.
Training can teach deep learning networks to correctly label images of cats in a limited set, before the network is put to work detecting cats in the
broader world.
Now you have a data structure and all the weights in there have been balanced based on what it has learned as you sent the training data through.
It’s a finely tuned thing of beauty. The problem is, it’s also a monster when it comes to consuming compute. Andrew Ng, who honed his AI
chops at Google and Stanford and is now chief scientist at Baidu’s Silicon Valley Lab, says training one of Baidu’s Chinese speech recognition
models requires not only four terabytes of training data, but also 20 exaflops of compute — that’s 20 billion billion math operations — across the
entire training cycle. Try getting that to run on a smartphone.

That’s where inference comes in.


Congratulations! Your Neural Network Is Trained and Ready for Inference
That properly weighted neural network is essentially a clunky, massive database. What you had to put in place to get that sucker to learn — in our
education analogy all those pencils, books, teacher’s dirty looks — is now way more than you need to get any specific task accomplished. Isn’t
the point of graduating to be able to get rid of all that stuff?

If anyone is going to make use of all that training in the real world, and that’s the whole point, what you need is a speedy application that can
retain the learning and apply it quickly to data it’s never seen. That’s inference: taking smaller batches of real-world data and quickly coming
back with the same correct answer (really a prediction that something is correct).

While this is a brand new area of the field of computer science, there are two main approaches to taking that hulking neural network and
modifying it for speed and improved latency in applications that run across other networks.

How Inferencing Works


How is inferencing used? Just turn on your smartphone. Inferencing is used to put deep learning to work for everything from speech recognition
to categorizing your snapshots.
The first approach looks at parts of the neural network that don’t get activated after it’s trained. These sections just aren’t needed and can be
“pruned” away. The second approach looks for ways to fuse multiple layers of the neural network into a single computational step.

It’s akin to the compression that happens to a digital image. Designers might work on these huge, beautiful, million pixel-wide and tall images,
but when they go to put it online, they’ll turn into a jpeg. It’ll be almost exactly the same, indistinguishable to the human eye, but at a smaller
resolution. Similarly with inference you’ll get almost the same accuracy of the prediction, but simplified, compressed and optimized for runtime
performance.
What that means is we all use inference all the time. Your smartphone’s voice-activated assistant uses inference, as does Google’s speech
recognition, image search and spam filtering applications. Baidu also uses inference for speech recognition, malware detection and spam filtering.
Facebook’s image recognition and Amazon’s and Netflix’s recommendation engines all rely on inference.

GPUs, thanks to their parallel computing capabilities — or ability to do many things at once — are good at both training and inference.

Systems trained with GPUs allow computers to identify patterns and objects as well as — or in some cases, better than — humans (see
“Accelerating AI with GPUs: A New Computing Model”).

After training is completed, the networks are deployed into the field for “inference” — classifying data to “infer” a result. Here too, GPUs — and
their parallel computing capabilities — offer benefits, where they run billions of computations based on the trained network to identify known
patterns or objects.

You can see how these models and applications will just get smarter, faster and more accurate. Training will get less cumbersome, and inference
will bring new applications to every aspect of our lives. It seems the same admonition applies to AI as it does to our youth — don’t be a fool, stay
in school. Inference awaits.

To learn more, check out NVIDIA’s inference solutions for the data center, self-driving cars, video analytics and more. Or to learn more
about the evolution of AI into deep learning, tune into the AI Podcast for an in-depth interview with NVIDIA’s own Will Ramey.

Real-time ray-tracing is the talk of the 2018 Game Developer Conference. So what is it? Where have you seen it before? And how does it
differ from rasterization? Check out “What’s the Difference Between Ray Tracing and Rasterization?”

Categories: Deep Learning | Explainer | The AI Podcast


Tags: Artificial Intelligence | GPU Computing
LOAD COMMENTS

LISTEN IN TO THE AI PODCAST


LISTEN NOW

ALL NVIDIA NEWS


AI Potcast: A Joint Discussion on AI, Agtech with Grownetics CEO
May 27, 2019
An On-Ramp for AI: NVIDIA Expands NGC-Ready Portfolio and Support Services
May 27, 2019
NVIDIA Kicks Off Computex, Supercharging Computers for Gamers and Creators, Driving AI at the Edge
May 27, 2019
World’s Most Powerful Mobile Workstations Launch with NVIDIA Quadro RTX, Delivering AI, Ray Tracing, VR
May 27, 2019
GPU Computing 101: Why University Educators Are Pulling NVIDIA Teaching Kits into Their Classrooms
May 24, 2019

Post navigation

Platforms

 CUDA-X
 Autonomous Machines
 Data Center
 Deep Learning and AI
 Design and Visualization
 Healthcare
 High Performance Computing
 Self-Driving Cars
 Gaming & Entertainment
 NGC

Products

 DGX Systems
 DRIVE PX
 GeForce rTX 20-Series
 NVIDIA Virtual GPU
 Jetson
 Quadro
 SHIELD TV
 Tesla
 T4 Enterprise Server

Developers

 NVIDIA Developer
 Developer News
 Developer Blog
 Developer Forums
 Open Source Portal
 Training
 GPU Tech Conference
 CUDA

Corporate

 NVIDIA Partner Network


 Careers
 Contact Us
 Security
 Communities
 RSS Feeds
 Email Signup
 Privacy Center
 Share Your Story Idea
EXPLORE OUR REGIONAL BLOGS AND OTHER SOCIAL NETWORKS

 © 2019 NVIDIA

 Legal Info

 Privacy Policy

You might also like