Lustre File System

chris
Outlines
 What is lustre
 How does it works
 Features
 Performance
What is Lustre?
 Cluster File Systems, Inc. (CFS) is the company
that originally developed the Lustre distributed file
system.

 On September 12, 2007, Sun Microsystems, Inc.

signed an agreement to acquire CFS.[1] The
acquisition was completed on October 1, 2007.

 Lustre is a shared disk file system, generally used

for large scale cluster computing.

 The name Lustre is a portmanteau of Linux and

cluster. The project aims to provide a file system
for clusters of tens of thousands of nodes with
petabytes of storage capacity, without
compromising speed or security. Lustre is
available under the GNU GPL.
Support Platform
Lustre Architecture
Lustre Components
 MGS
 The Management Server (MGS) defines
configuration information for all Lustre
filesystems at a site.

 Each Lustre target contacts the MGS to

provide information, and Lustre clients
contact the MGS to retrieve information.

 The MGS can provide live updates to the

configuration of targets and clients. The
MGS requires its own disk for storage.
However, there is a provision that allows the
MGS to share a disk ("co-locate") with a
single MDT.
Lustre Components
 MDT
 The MDT provides back-end storage for
metadata for a single filesystem. The
Metadata Server (MDS) provides the
network request handling for one or more
local MDTs.

 The metdata managed by the MDT consists of

the file hierarchy ("namespace"), along with
file attributes such as permissions and
references to the data objects stored on the
OSTs.
Lustre Components
 OSS and OSTs
 An OST provides back-end storage for file object
data (effectively, chunks of user files). Typically,
multiple OSTs provide access to different file
chunks.

 The MDT tracks the location of the chunks. On a

node serving OSTs, an Object Storage Server
(OSS) component provides the network request
handling for one or more local OSTs.
Lustre Configuration
1. Create Lustre format file system on
MGS/OSS
 mkfs.lustre
 --fsname={whatever}
 --{--ost/--mdt/--mgs}
 --mgsnode={hostname@tcp0}
 /dev/sda
2. Mount lustre filesystem on MGS and OSS
 mount –t lustre /dev/sda /mnt/test
 mount mgs/mdt at first
3. Mount lustre filesystem on clients
 mount –t lustre {hostname@tcp0}:/{fsname}
/mnt/client_FS
Lustre Configuration
Lustre Failover mechanism
 Active/active
 Active/passive
 Heartbeat
 STONITH (Shoot The Other Node In The Head

Heartbeat msg OSS-

1-
OSS-1
backu
STONITH p
poweroff
Lustre Failover mechanism
 In Lustre, a disk failure is an unrecoverable
error. For this reason, you must have reliable
back-end storage with RAID.
 Backup device
 dd if=/dev/{original} of=/dev/{new} bs=4k
conv=sync,noerror
Features
 Kerberos 、 ACL
 Bonding
 Live Add/Remove OSTs
 SNMP
 Quota
 File Stripe size
 Using POSIX Standard
Performance
 Read/write files with different kernel io scheduler
Performance
 Create files with different Kernel io scheduler
Performance
 Multi-clients
Performance
 Lustre vs GPFS
Reference
1. Sun Microsystems Expands High Performance Co
, press release, September 12, 2007.
2. Lustre™ 1.6 Operations Manual
3. http://wiki.lustre.org/index.php?
title=Main_Page
Any Questions?