BIG DATA

An Introduction

DATA
What is Data?
Information in raw or unorganized form (such as alphabets, numbers, or symbols) that refer to, or represent, conditions, ideas, or objects.

Distinct pieces of information, usually formatted in a special way. A collection of facts, such as values or measurements.
Data can exist in a variety of forms -- as numbers or text on pieces of paper, as bits and bytes stored in electronic memory, or as facts stored in a person's mind.

Few Examples
A word file, an excel sheet, an image, an e-mail, a Facebook status, a Twitter tweet, a YouTube video and the list is endless.

AROUND THE WORLD IN 60secs.

EVOLUTION OF COMPUTERS
1960 Main Frame Computers
Took large space. Occupied an entire room. All data centralized. Stored at a single location. To analyze the data stored, physicists had to come from across the globe. No networks. Difficult management of data.

1970 IBM Clones

Comparatively less space. Occupies an entire rack. De-centralization of data starts. Mainframes moduled into small units, data distributed across them. Modules connected with network bridges. Computers become feasible.

1980-90 Desktop Computers

Get arranged simply on a desk. Internet expands and becomes global, network protocols developed. Ease of data communication and sharing. Access from remote locations. Amount of data increases in many folds. Still in use widely in offices, institutes etc.

EVOLUTION OF COMPUTERS
Late 90s-2000 Laptop Computers
Computers become portable and handy-mobile. Commonly used in a variety of settings, including work, education, and personal multimedia. A laptop combines the components and inputs as a desktop computer; including display, speakers, keyboard, and pointing device. Modern day laptops are equipped with connecting and networking technologies such as Bluetooth, WI-FI etc. In wide use across the globe.

2010 Tablet Computers

A tablet computer, or simply tablet, is a mobile computer with display, circuitry and battery in a single unit. Tablets are equipped with sensors, including cameras, microphone, accelerometer and touch screen, with finger or stylus gestures replacing computer mouse and keyboard. Used widely for networking purposes.

EVOLUTION OF COMPUTERS

Main Frames

IBM Clones

Desktop PC

Laptop

Tablet

And the story still continues..

THE INTERNET
Technologists say, the world has turned to a Global Village. Well, the credit to this goes to the Internet. The Internet has made distances shorter and the world smaller. The Internet is defined as the worldwide interconnection of individual networks operated by government, industry, academia, and private parties. In a matter of very few years, the Internet consolidated itself as a very powerful platform that has changed forever the way we do business, and the way we communicate. Size of Internet Expanding Daily. Every hour, every minute, rather every second.

PRESENT SCENARIO

INTERNET USERS WORLDWIDE

Latest numbers 2749 millions (source- itu mar13 report)

PRESENT SCENARIO

SMART-PHONE & TABLET USERS - WORLDWIDE

THE PROBLEM
The core problem going by todays scenario is the extensively growing quantum of Data.

The world's technological per-capita capacity to store information has roughly doubled every 40 months since the 1980s. From the beginning of human history till 2003 only 5 exabytes of data was generated. But since then, the situation has transformed drastically. Everyday we create 2.5 quintillion bytes of data; 90% of the data in the world today has been created in the last two years alone. 1 Quintillion = 10^18 bytes i.e 1000000000000000000 bytes or 1 exabyte. Too much.. Right? Predictions say that the same amount of data would be generated in minutes in the days to come. The Big Question How to manage & process this enormous data store using the present technologies?

THE PROBLEM
Few Illustrations How Much Data?

Google processes 20 PB a day Wayback Machine has 3 PB + 100 TB/month Facebook has 2.5 PB of user data + 15 TB/day eBay has 6.5 PB of user data + 50 TB/day CERNs Large Hydron Collider (LHC) generates 15 PB a year

Sensor technology and networks (measuring all kinds of data)

Mobile devices (tracking all objects all the time)

Social media and networks (all of us are generating data)

Sensor technology and networks (measuring all kinds of data)

BIG DATA A SOLUTION?

What is Big Data?
It is an elusive concept. Represents the collection of digital data that is uncomfortable to store, transport and analyze. Big data is the term for a collection of data sets so large and complex that it becomes difficult to process using on-hand database management tools or traditional data processing applications. The challenges include capture, curation, storage, search, sharing, transfer, analysis, and visualization. Challenges us to create next-gen data storage tools and techniques. Old concept- Few companies generate data, all others consume it. New concept- All of us are generating data, and all of us are consuming the same.
Big Data usually includes data sets with sizes beyond the ability of commonly used software tools to capture, curate, manage, and process the data within a tolerable elapsed time.

BIG DATA THE 4 Vs

BIG DATA KEY APPLICATIONS

So, get prepared for the future of data storage!!

Thank You!