MSDN Event

WINDOWS AZURE STORAGE

Windows Azure Storage

Storage in the Cloud
Scalable, durable, and available Anywhere at anytime access Only pay for what the service uses

Exposed via RESTful Web Services

Use from Windows Azure Compute Use from anywhere on the internet

Various storage abstractions

Tables, Blobs, Queues, Drives
3

The Storage Client API

The Main API are RESTfuls
Can call these from any HTTP client e.g. Flash, Silverlight, etc

For easier .NET access, a Client API from SDK Microsoft.WindowsAzure.StorageClient

Provides a strongly typed wrapper around REST services

Windows Azure Storage Account

User specified globally unique account name
Can choose geo-location to host storage account
US North Central and South Central
Europe North and West Asia East and Southeast

Can CDN Enable Account

Blobs delivered via 18 global CDN nodes

Can co-locate storage account with compute account

Explicitly or using affinity groups

Accounts have two independent 512 bit shared secret keys

100TB per account
5

Storage in the Development Fabric

Provides a local Mock storage

Emulates storage in cloud

Allows offline development Requires SQL Express 2005/2008 or above
There are some differences between Cloud and Dev Storage. http://msdn.microsoft.com/dd320275 A good approach for developers: To test pre-deployment, push storage to the cloud first Use Dev Fabric for compute connect to cloud hosted storage. Finally, move compute to the cloud. 6

Windows Azure Storage Abstractions

Blobs

Drives

Tables
Provide structured storage. A Table is a set of entities, which contain a set of properties

Queues

Windows Azure Storage Account

Blob Storage Concepts

http://<account>.blob.core.windows.net/<container>/<blobname>

Account

Container

Blob

PIC01.JPG

images
PIC02.JPG cohowinery

videos
9

VID1.AVI

Blob Features and Functions

PutBlob GetBlob DeleteBlob CopyBlob SnapshotBlob LeaseBlob

<name, value>

10

Blob Client Library

CloudStorageAccount account = CloudStorageAccount.FromConfigurationSetting("CloudStorageAccount"); CloudBlobClient blobClient = new CloudBlobClient( account.BlobEndpoint, account.Credentials); // Create Container CloudBlobContainer cloudContainer = blobClient.GetContainerReference(containerName); bool hasCreated = cloudContainer.CreateIfNotExist(); // Access Blob in the Container CloudBlob cloudBlob = cloudContainer.GetBlobReference(blobName); //BlobRequestOptions has retry policy, timeout etc. BlobRequestOptions options = new BlobRequestOptions(); //Upload the local file to Blob service cloudBlob.UploadFile(uploadFileName, options); //Download to local file name cloudBlob.DownloadToFile(downloadFileName, options); 11

Two types of Blobs under the hood

blocks

pages

12

Block Blob
Streaming Workload w/ Random Reads + Committed Writes

10 GB Movie
Block Id 1 Block Id 2 Block Id 3 Block Id N

blobName = blob.wmv; PutBlock(blobName, blockId1, block1Bits); PutBlock(blobName, blockId2, block2Bits); PutBlock(blobName, blockIdN, blockNBits); PutBlockList(blobName, blockId1, blockId2,blockIdN);

blob.wmv blob.wmv

Windows Azure Storage

13

Page Blob Random Read/Write

0 512 1024 1536 2048 2560

10 GB Address Space
14

PutPage 512, 2048 PutPage 0, 1024

ClearPage 512, 1536

PutPage 2048,2560

GetPageRange 0, 4096
0,512 1536,2560

GetBlob 1000, 2048

10 GB

1536,2048

Windows Azure Content Delivery Network

15

Blobs Tips
ServicePointManager.DefaultConnectionLimit

ParallelOperationThreadCount

CloudBlobClient

BlobRequestOptions Timeout

16

Windows Azure Drive

17

Windows Azure Drives Capabilities

18

Cloud Library Sample

//Create Local Storage resource and initialize the local cache for drives CloudDrive.InitializeCache(localCacheDir, cacheSizeInMB);
CloudStorageAccount account = CloudStorageAccount.FromConfigurationSetting("CloudStorageAccount");

//Create a cloud drive (PageBlob) CloudDrive drive = account.CreateCloudDrive(pageBlobUri); drive.Create(1000 /* sizeInMB */);
//Mount the network attached drive on the local file system string pathOnLocalFS = drive.Mount(cacheSizeInMB, DriveMountOptions.None); //Use NTFS APIs to Read/Write files to drive //Snapshot drive while mounted to create backups Uri snapshotUri = drive.Snapshot(); //Unmount the drive drive.Unmount(); 19

Windows Azure Tables

20

Table Storage Concepts

Account Table Entity
Name = Email = customers Name = Email = cohowinery Photo ID = Date = winephotos Photo ID = Date = 21

Entity Properties
PartitionKey & RowKey

Timestamp

<name, typed value>

String, binary, bool, DateTime, GUID, int, int64, and double

22

PartitionKey and Partitions

23

Purpose of the Partition Key

24

Partitions and Partition Ranges

PartitionKey PartitionKey (Category) (Category) RowKey RowKey (Title) (Title) Fast & Furious Fast & Furious The Bourne Ultimatum The Bourne Ultimatum Open Season 2 Open Season 2 The Ant Bully The Ant Bully RowKey Office Space (Title) Timestamp Timestamp Timestamp ReleaseDate ReleaseDate 2009 2009 2007 2007 2009 2009 2006 2006 ReleaseDate 1999

Server A
Table = Movies
[MinKey - Comedy)

Action Action Action Action Animation Animation Animation Animation

PartitionKey (Category) Comedy

Server A
Table = Movies

Server B
Table = Movies
[Comedy - MaxKey)

Comedy SciFi SciFi

War War
25

Office Space X-Men Origins: Wolverine X-Men Origins: Wolverine

Defiance Defiance

1999 2009 2009

2008 2008

Table Operations
Create, Query, Delete

Insert Update
Merge Replace

Delete Query Entity Group Transactions

26

Hello Azure Architecture

Client

Client

TwitterAggragator

Table Storage Hello Azure Web App

Aggregator Instance 0 Web Role

Aggregator Instance 1

Cloud
Twitter TwitterDispathcer Web Role

Client

Dispatcher Instance 0

Queue

Dispatcher Instance 1

27

DEMO

Twitter Table

28

Table Tips
Scale
Queries Entity Group Transactions

Avoid Append only write patterns based on PartitionKey values

Avoid using monotonically increasing suffix with a constant prefix
Example: using only the current timestamp as PartitionKey

If needed, add varying prefix to PartitionKey

29

Table Tips cont.

Avoid large table scans when performance is critical
Restructure your schema if required Concatenate different keys to form appropriate index Most Optimal:

PartitionKey == SciFi and RowKey == Star Wars

Scans: Expect continuation tokens (REST) PartitionKey == SciFi and Sphere RowKey Star Wars Action PartitionKey Thriller

PartitionKey == Action || PartitionKey == Thriller - currently scans entire table

Cars RowKey Star Wars - scans entire table

30

Queue Storage Concepts

Account

Queue

Message

customer ID order ID http://

cohowinery order processing customer ID order ID http://

31

Loosely Coupled interaction with Queues

Input Queue (Work Items)

Azure Queue

32

Queues Reliable Delivery

Input Queue (Work Items)

Azure Queue

Dequeues Deletes

33

DEMO

Twitter Queue

34

Queue Tips
8KB

DequeueCount

35