Space Management

Module 12 Data ONTAP 8.0 7-Mode Administration

Module Objectives
By the end of this module, you should be able to: List the factors that impact space consumption in Data ONTAP Describe how and when a volume consumes space from its containing aggregate Explain how to guarantee writes for a file Discuss how Data ONTAP can provide more space to a full volume State deduplication techniques available in Data ONTAP
2009 NetApp. All rights reserved.

Factors

2009 NetApp. All rights reserved.

Space Management Factors

Administrators have flexibility to manage their storage systems by allocating volumes as: Full provisioned volumes (Space guarantee)
Requires reserving space within the aggregate for the volume at volumes creation Default allocation Cannot over-commit an aggregate Simple storage management

Thin provisioned volumes (Non-space guarantee)

Does not require reserving space within the aggregate for the volume at the volumes creation Allows more aggressive allocation May over-commit an aggregate Complex storage management
2009 NetApp. All rights reserved.

Space Management Factors (Cont.)

Within a volume, whether full or thin provisioned, all files, whether a LUN or a NAS file, may:
Have space reservations (default for LUNs)
Requires reserving space within the volume so the entire file may be overwritten even if blocks are retained by a Snapshot copy Writes to the file will succeed Simpler management

Not have space reservations (default for NAS file)

Does not require reserving space within the volume Writes to the file might fail Active space monitoring required

See the SAN Administration on Data ONTAP course for more information on LUN reservations
2009 NetApp. All rights reserved.

Volume Space Guarantee

2009 NetApp. All rights reserved.

Space Guarantees
Space guarantee is an attribute of a volume that reserved or set aside out of the containing aggregate Space guarantee parameters: volume (default): Reserves the FlexVol total size within its containing aggregate and allows for space reserved or nonspace reserved files file: Allow the creation of a space guaranteed file total size within its containing FlexVol volume none: Does not reserves any space for the FlexVol volume within its containing aggregate and allows only for non-space guaranteed files Space guarantee may be set: At the volumes creation: vol create -s volume vol1 aggr1 5GB On existing volumes: vol options vol1 guarantee none
2009 NetApp. All rights reserved.

System Manager: Space Guarantees

To configure volumes

2009 NetApp. All rights reserved.

Space Guarantee Example

Create a five-disk aggregate:
aggr create aggr1 5 aggr1

df -Ag aggr1
Total AFS (95%) SSR (5%)
85 4

Used
0 0

Avail
85 4

Icon

Description
aggr1
72-GB disk

2009 NetApp. All rights reserved.

Space Guarantee Example (Cont.)

Create a 30 GB volume:
vol create vol1 aggr1 30g aggr1

df -Ag aggr1
Total AFS (95%) SSR (5%)
85 4

Used
30 0

Avail
54 4

df -g vol1
Total Used
0 0

Avail
24 6

AFS (80%)
SSR (20%) Icon

24 6

vol1

Description
aggr1
72-GB disk

Default is volume space guarantee

2009 NetApp. All rights reserved.

Space Guarantee Example (Cont.)

Create a 30 GB vol2:
vol create vol2 -s none aggr1 30g aggr1

df -Ag aggr1
Total AFS (95%) SSR (5%)
85 4

Used
30 0

Avail
54 4

vol2

df -g vol2
Total Used
0 0

Avail
24 6

AFS (80%)
SSR (20%) Icon

24 6

vol1

Description
aggr1
72-GB disk

With none space guarantee

2009 NetApp. All rights reserved.

Space Guarantee Example (Cont.)

Write 24 GB to vol2
Space taken from aggregate when written

aggr1

df -Ag aggr1
Total AFS (95%) SSR (5%)
85 4

Used
54 0

Avail
30 4

vol2

df -g vol2
Total Used
24 0

Avail
0 6

AFS (80%)
SSR (20%) Icon

24 6

vol1

Description
aggr1
72-GB disk

With none space guarantee

2009 NetApp. All rights reserved.

File Types
Files, within volumes, may be either:
Normal: The size of the file represents the amount of data within the file
foo Physical data

Sparse: The size of the file is greater than the amount of data within the file
foo2

2009 NetApp. All rights reserved.

File Type Example

Create a 30 GB vol3:
system> vol create vol3 -s file aggr1 30g

aggr1

df -Ag aggr1
Total AFS (95%) SSR (5%)
85 4

vol3
Used
54 0

Avail
30 4

vol2

df -g vol3
Total Used
0 0

Avail
24 6

AFS (80%)
SSR (20%) Icon

24 6

vol1

Description
aggr1
72-GB disk

With file space guarantee

2009 NetApp. All rights reserved.

File Type Example (Cont.)

Create a 10 GB file:
# dd if=/dev/zero of=foo bs=1000 count=10000000

aggr1

df -Ag aggr1
foo

vol3

Total AFS (95%) SSR (5%)

85 4

Used
64 0

Avail
20 4

vol2

df -g vol3
Total Used
10 0

Avail
14 6

AFS (80%)
SSR (20%) Icon

24 6

vol1

Description
aggr1
72-GB disk

With file space guarantee

2009 NetApp. All rights reserved.

File Type Example (Cont.)

Create a 10 GB sparse file:
# dd if=/dev/zero of=foo2 bs=1 count=0 seek=10000000

aggr1

df -Ag aggr1
foo

vol3

foo2

Total AFS (95%) SSR (5%)

85 4

Used
74 0

Avail
10 4

vol2

df -g vol3
Total Used
10 0

Avail
14 6

AFS (80%)
SSR (20%) Icon

24 6

vol1

Description
aggr1
72-GB disk

With file space guarantee

2009 NetApp. All rights reserved.

Solutions to Full Volumes

2009 NetApp. All rights reserved.

Solutions for Full Volumes

If a volume fills up, because of Snapshot copies, active file system data, or both Then administrators can:
Delete Snapshot copies manually or automatically Expand the volume Delete active file system data if the blocks are not part of a Snapshot copy

2009 NetApp. All rights reserved.

Snapshot Automatic Delete

Snapshot automatic delete determines when (if) Snapshot copies will be automatically deleted
Set at volume level snap autodelete vol[on|off|show|reset]

If autodelete is enabled, then options:

snap autodelete vol options option val Options commitment trigger target_free_space delete_order defer_delete prefix
2009 NetApp. All rights reserved.

Value try, disrupt volume, snap_reserve, space_reserve 1-100 oldest_first, newest_first scheduled, user_created, prefix, none <string>

Volume Autosize
Might want to grow the volume vol autosize determines if a volume should grow when nearly full
Set at volume level Possible values:
ON
Increment size (default 5% of original size) Maximum size (default 120% of original size)

OFF
vol autosize vol_name [-m size[k|m|g|t]] [-i size[k|m|g|t]] [on|off|reset]

2009 NetApp. All rights reserved.

Administrators Choice
Administrators may choose which procedure to employ first:
snapshot auto delete vol autosize Use the volume option:
try_first Possible values:
snap_delete volume_grow (default)

Example:
vol options vol_name try_first snap_delete

2009 NetApp. All rights reserved.

Deduplication

2009 NetApp. All rights reserved.

Deduplication
NetApp Deduplication

NetApp deduplication
20:1 or greater for backup

Integrated with Data ONTAP

General-purpose volume deduplication Identifies and removes redundant data blocks

Application agnostic
Before After Primary storage Backup data Archival data

Service
Runs as a background process and is transparent to any client

2009 NetApp. All rights reserved.

Deduplication in Action
presentation.ppt = Identical blocks Original file 20 blocks presentation.ppt With NetApp deduplication 30 total blocks

Identical file 20 blocks presentation.ppt

Without NetApp deduplication 70 total blocks

Edited file 10 blocks added

2009 NetApp. All rights reserved.

NetApp Deduplication: Internals

Initialization (only necessary on pre-existing volume) Gather

Gatherer File

Gathering

qsort

qsort

...

qsort

Merge Sort Sorting

Fingerprint File

2009 NetApp. All rights reserved.

NetApp Deduplication: Internals (Cont.)

Byte-by-byte comparison Count file update Update inode

Sort by Inode

Update Inode

Duplicate Entry File qsort qsort ... qsort

Block Ref Count File

Merge Sort

Fingerprint File

2009 NetApp. All rights reserved.

NetApp Deduplication: Internals (Cont.)

Block Write

Log New FPs

Change Log File

Change Log File

Fingerprint File

2009 NetApp. All rights reserved.

NetApp Deduplication: Internals (Cont.)

Block Write

Log New FPs

Change Log File

Change Log File

qsort

qsort

...

qsort

Merge Sort

Fingerprint File

2009 NetApp. All rights reserved.

NetApp Deduplication: Internals (Cont.)

Block Write Byte-by-byte comparison Count file update Update inode

Log New FPs

Change Log File

Change Log File

Sort by Inode

Update Inode

Duplicate Entry File qsort qsort ... qsort

Block Ref Count File

Merge Sort

Fingerprint File

2009 NetApp. All rights reserved.

NetApp Deduplication: Internals (Cont.)

Block Write

Log New FPs

Change Log File

Change Log File

Fingerprint File

SIS Check

2009 NetApp. All rights reserved.

NetApp Deduplication: Stages

Initialization (only necessary on pre-existing volume) Gather Block Write Byte-by-byte comparison Count file update Update inode

Log New FPs

Gatherer File

Change Log File

Change Log File

Sort by Inode

Update Inode

Gathering

Duplicate Entry File qsort qsort ... qsort Deduplicating

Block Ref Count File

Merge Sort Sorting

Fingerprint File

SIS Check Checking

2009 NetApp. All rights reserved.

Configuration Overview
License it:
system> license add <license>

Turn it on:
system> sis on <vol>

Deduplicates existing data:

system> sis start -s <vol>

Schedule when to deduplicate or run manually:

system> sis config [-s schedule] <vol> system> sis start <vol>

For maintenance:
system> sis status [-l] <vol> system> sis check

View the space savings:

system> df s <vol>
2009 NetApp. All rights reserved.

Configuring Deduplication
system> sis on /vol/vol1
SIS for "/vol/vol1" is enabled. Already existing data could be processed by running "sis start -s /vol/vol1".

system> sis start -s /vol/vol1

The file system will be scanned to process existing data in /vol/vol1. This operation may initialize related existing metafiles. Are you sure you want to proceed with scan (y/n)? y Fri Nov 10 11:42:58 EST [wafl.scan.start:info]: Starting SIS volume scan on volume vol1. The SIS operation for "/vol/vol1" is started.

2009 NetApp. All rights reserved.

Configuring Deduplication (Cont.)

system> sis status /vol/vol1
Path /vol/vol1 ... Enabled State Active Status 12 GB Progress Scanned

system> sis status /vol/vol1

Path /vol/vol1 State Enabled Status Idle Progress Idle for 00:01:26

system> df -s /vol/vol1
Filesystem /vol/vol1 used 20568268 saved 3768732 %saved 15%

2009 NetApp. All rights reserved.

sis status Progress and Stages

system> sis status
Path State Enabled Status Progress Active 25 MB Scanned

Gathering

/vol/vol1

Sorting

Path /vol/vol1

State Enabled

Status Progress Active 25 MB Searched

Deduplicating

Path /vol/vol1

State Enabled

Status Progress Active 40MB (20%) done

Path

State Enabled

Status Progress Active 30MB Verified

Checking

/vol/vol1 OR /vol/vol1

Enabled

Active

10% Merged

2009 NetApp. All rights reserved.

Scheduling Deduplication
Default schedule:
system> sis on /vol/vol1 system> sis status /vol/vol1 sun-sat@0

To configure a schedule:
system> system> system> system> sis sis sis sis config config config config -s -s s s - /vol/vol1 23@sun-fri /vol/vol1 auto /vol/vol1 sat@6 /vol/vol1

2009 NetApp. All rights reserved.

Other Commands
vol status command
SIS keyword will be listed in the output for deduplication volumes system> vol status Volume State Vol0 online Vol1 online

Status raid_dp, flex raid_dp, flex

Options root sis

2009 NetApp. All rights reserved.

System Manager: Deduplication

Start the dedupe

Deduplication license was added

2009 NetApp. All rights reserved.

Module Summary
In this module, you should have learned to: List the factors that impact space consumption in Data ONTAP Describe how and when a volume consumes space from its containing aggregate Explain how to guarantee writes for a file Discuss how Data ONTAP can provide more space to a full volume State deduplication techniques available in Data ONTAP
2009 NetApp. All rights reserved.

Exercise
Module 12: Space Management Estimated Time: 45 minutes

Check Your Understanding

What is a fully provisioned volume?
A volume where space is taken from the containing aggregate at the moment of volume creation

What is a thin provisioned volume?

A volume where space is taken from the containing aggregate at the moment space is allocated to files

Why would you use thin provisioned volumes?

Allocating space to users that may use it is not efficient; use to over-allocate an aggregate to maximum space usage
2009 NetApp. All rights reserved.