You are on page 1of 4

Hadoop 1

2013-07-23

Google Google 5

GoogleClusterChubbyGFSMapReduceBigTable

googlecluster-iee chubby-osdi06.pdf gfs-sosp2003.pdf


e.pdf

MapReduce.pdf

bigtable-osdi06.p
df

03 06 03
SOSP GFS04 OSDI MapReduce 06 OSDI BigTable

SOSP OSDI A SOSP


OSDI

Apache Apache
Hadoop

Chubby-->ZooKeeper GFS-->HDFS

BigTable-->HBase MapReduce-->Hadoop MapReduce


Facebook HiveYahho Pig
Hadoop Hadoop
Hadoop

Hadoop
Hadoop PB Hadoop

Hadoop
Hadoop
Hadoop
1

Hadoop

Hadoop

Hadoop

Hadoop Java Linux


Hadoop C++
Apache Hadoop
Apache Cassandra
Hadoop
Hbase
Hive Hadoop
Apache Pig
Apache ZooKeeper

Chukwa

Apache Mahout Hadoop

Hadoop HDFS MapReduce


HDFSMapReduce Hadoop
Hadoop
Apache12 October, 2012: Release 1.0.4 available
Hadoop core/mapred/tools/hdfs

Hadoop Hadoop
HDFS API
2

Amazon S3

confconf fs
fs

Hadoop
Package

Dependences

tools

DistCparchive

mapreduce

Hadoop Map/Reduce

filecache

HDFS Map/Reduce

fs

hdfs

HDFSHadoop

ipc

IPC io

io

net

DNSsocket

security

conf

metrics

util

record

DDL
C++ Java

http

Jetty HTTP Servlet

log

HTTP HTTP Servlet