Solutions Overview

NetApp Media Content Management Solution Based on E-Series Solutions Guide

Bob Houser, TESO E-Series Team, NetApp September 2011 | TR-3957

TABLE OF CONTENTS

INTRODUCTION .................................................................................................................................... 3
1.1 1.2 NETAPP MEDIA CONTENT MANAGEMENT SOLUTION BENEFITS ...................................................................... 3 TARGET AUDIENCE ................................................................................................................................................. 4

SOLUTION SUMMARY.......................................................................................................................... 5
2.1 SUPPORT DOCUMENTS .......................................................................................................................................... 6

SOLUTION ARCHITECTURE ................................................................................................................ 7

3.1 3.2 3.3 HARDWARE REQUIREMENTS................................................................................................................................. 7 SOFTWARE REQUIREMENTS ................................................................................................................................. 8 SOLUTION ARCHITECTURE DETAILS .................................................................................................................... 9

NETAPP MEDIA CONTENT MANAGEMENT SOLUTION ................................................................. 11

4.1 4.2 4.3 4.4 INTEGRATED SOLUTION TESTING AND OBJECTIVES ....................................................................................... 11 STORNEXT FMS CONFIGURATION FILE (TEST SCENARIO) .............................................................................. 11 STORNEXT CLIENT CONFIGURATIONS ............................................................................................................... 11 THE E5460 TEST RESULTS ................................................................................................................................... 12

SUMMARY ........................................................................................................................................... 13

LIST OF TABLES Table 1) Test case results: buffered reads/writes, 100% sequential, 24 total streams, E5460. ................. 13

LIST OF FIGURES Figure 1) Typical MCM workflow. ................................................................................................................. 4 Figure 2) E-Series MCM solution. ................................................................................................................. 6 Figure 3) E5460 with front drawer extended and E5460 components. ........................................................ 8 Figure 4) SAN and LAN layout. .................................................................................................................... 9 Figure 5) Cross-connected SAN. ................................................................................................................ 10

NetApp Media Content Management Solution Based on E-Series Solutions Guide

1 INTRODUCTION
The commercial video industry is constantly improving video formats and transport technologies. These improvements result in faster editing and larger storage requirements, along with the need to produce video images in larger capacity forms. From production to postproduction, broadcast, and distribution, the industry is recognizing the benefits of digital technology, improved media formats, and the subsequent increased storage requirements. In addition, this shift has fueled the explosive growth of high-bandwidth distribution through the Internet and private networks, offering consumers ready access to a broad spectrum of high-definition and big-data content. The expanding volume and size of media content result from the exponential increase in high-resolution formats used by audio, video, and high-end graphics. Additionally, more of this content is kept accessible and online longer. Providing immediate access to archived files is becoming common. All of these factors mean that companies face serious challenges with respect to storage density and timely access. Traditional storage architectures are not designed for todays workflow and for the distribution of the large files generated by video media enterprises and the entertainment industry. To stay competitive, business and technical processes must constantly improve to enable creative collaboration and to provide costeffective distribution methods. This environment needs a shared storage pool that can expand to accommodate changing capacity requirements, provide continuous and timely access to video, and deliver efficiencies that enable greater productivity. Moving to a centralized, shared repository enables media- and video-centric companies to consolidate storage silos, eliminate unnecessary redundancy, provide fast data access, and simplify management.

1.1

NETAPP MEDIA CONTENT MANAGEMENT SOLUTION BENEFITS

The NetApp Media Content Management (MCM) Solution Based on E-Series focuses on managing media content. This solution is targeted mainly at broadcasters, media studios, and media content aggregators. It offers customers a centralized media repository that is massively scalable, for both capacity and performance, to provision for their ingest, content creation, and distribution workflows. This solution enables collaborative workflows, shared data access, and increased efficiency. The NetApp ESeries is the optimal choice for this MCM solution because these environments require dense, scalable capacity, and high performance across a mix of large-block streaming and small-block random input/outputs (I/Os), combined with ease of use and high levels of reliability. Quantum StorNext provides a large single data container, performance aggregation, heterogeneous shared data access, and archive and data management capabilities. The NetApp MCM solution consists of the E5460 storage system, which features dual E5400 RAID controllers, each with 4-8Gb Fibre Channel (FC) connections and 12GB of cache, in the DE6600 4U 60drive enclosure, with 60-3TB NL-SAS drives, and StorNext File system v. 4.0.x. NetApp Professional Services and SupportEdge are required solution components. Optional components of the solution include a 40U rack, 4 additional 8Gb FC ports per E5460 controller, additional StorNext SAN Clients, StorNext Distributed LAN Clients, StorNext Storage Manager, and StorNext options for high availability and remote replication. The E5460 system offers the following features: Bandwidth easily handles the rigors of heavy throughput and bandwidth-sensitive streaming environments for both ingest and playout. Reliability of preconfigured systems results in fast and confident deployments, and the advanced thermal, power, and failover features prevent downtime. Standard components offer redundancy, delivering the utmost availability of critical data. Manageability offered by integration with the Quantum StorNext File System delivers streamlined management for large datasets. Scalability accommodates growing data streams, longer retention policies, and access requirements.

NetApp Media Content Management Solution Based on E-Series Solutions Guide

A generic MCM solution is composed of the following components: 1. Disk storage repository 2. Parallel file system 3. Ingest film and/or video streams 4. Editing, quality assurance, and/or playback workstations 5. Media archive and/or asset solution 6. Encoding and/or translation servers 7. Playout services and video on demand
Figure 1) Typical MCM workflow.

1.2

TARGET AUDIENCE

This MCM solution is recommended for the following audience: Media content aggregators and distribution companies Broadcasters and networks Film studios Multimedia corporations Sports leagues

NetApp Media Content Management Solution Based on E-Series Solutions Guide

2 SOLUTION SUMMARY
The NetApp E5460 coupled with the Quantum StorNext file system for management offers a scalable, high-performance shared storage system that is optimized for storing, sharing, and serving large numbers of high-definition media files. This solution is optimal for workflows that require: Large centralized storage capacity Non-real-time ingest and playout of high-bandwidth media streams A limited number of transcoding clients A limited number of edit stations High-resolution media asset management Online media archiving Centralized storage to provision content creation workstations and to broadcast playout workstations

A typical starting MCM solution system, shown in Figure 2, is composed of the E5400, a StorNext highperformance file system, and application workstations that process and provision video and a variety of media formats.

NetApp Media Content Management Solution Based on E-Series Solutions Guide

Figure 2) E-Series MCM solution.

2.1

SUPPORT DOCUMENTS

For more information on this solution, refer to the following documents: TR-3952: NetApp Media Content Management Solution Based on E-Series Sizing Guide Quantum StorNext File System Installation Guide http://downloads.quantum.com/SNMS/4.1/6-00360-23_StorNext_Install_Guide.pdf Quantum StorNext File System Tuning Guide http://downloads.quantum.com/SNMS/4.1/6-01376-18_TuningGuide.pdf Quantum StorNext File System Users Guide http://downloads.quantum.com/SNMS/4.1/6-01658-14_SN4.1_UserGuide.pdf SANtricity ES Storage 10.80 Installation Guide https://now.netapp.com/NOW/knowledge/docs/e-series_storage_systems/relfmv1/pdfs/51887-00.pdf

NetApp Media Content Management Solution Based on E-Series Solutions Guide

SANtricity ES Storage Concept Guide https://now.netapp.com/NOW/knowledge/docs/e-series_storage_systems/relfmv1/pdfs/51891-00.pdf

3 SOLUTION ARCHITECTURE
3.1 HARDWARE REQUIREMENTS

The MCM combines the NetApp E5460 with a high-performance file system, Quantum StoreNext. Each system provides 60 SAS 3.5-inch drives in a 4U drive enclosure, for a total of 180GB of raw disk per system. The system architecture requires a high-performance file system, Quantum StorNext 4.1 or later, and a minimum of two metadata controllers running on a Microsoft Windows or Linux platform. This packaged solution contains the major components needed for a content management application, an archival media environment, or other third-party media applications used in MCM environments. STORAGE SYSTEM BASE COMPONENTS The base components are: NetApp hardware: E-Series system (storage nodes) SKU: X-6600-0892-003-R6 Description: E5460A,60x3TB,Encl,Rails,Turbo,64-P 2 5400 controllers (FC base 24GB) DE6600 chassis 8 FC SFP 8 FC 2-meter cables Rail kit for DE6600 NetApp EOS controller firmware v. 10.80 or later, 64 partitions, turbo performance Disk drives POS: 60-3TB 7K rpm 3.5-inch NL-SAS disks in 6600 carrier Raw capacity (TB): 180 Desired MCM options StorNext (can be ordered through NetApp in the United States or directly from a Quantum representative) Power cord: Country specific

SKU: Order the following parts:

ADDITIONAL OPTIONAL COMPONENTS These components are optional: NetApp FAS or other server for StorNext metadata and journals 8Gb FC SAN switch (48 port) Emulex or ATTO HBAs Network management software Server management software Selected application management

NetApp Media Content Management Solution Based on E-Series Solutions Guide

SERVERS AND CLIENTS (CUSTOMER OR INTEGRATOR SUPPLIED) The servers and clients are: Mac , Microsoft Windows, and/or Linux servers and clients for media processing and editing x64-based Windows or Linux servers for StorNext management, metadata, and journals 2 HBAs per server, dual port 8GB FC recommended FC switches Check the NetApp compatibility matrix for equipment and software updates and requirements: http://now.netapp.com/matrix/mtx/login.do.

Note:

As shown in Figure 3, the NetApp E5460 is a 4U drive enclosure that includes 2 internal controllers and 60 NL-SAS disks accessible and hot swappable in 5 drawers.
Figure 3) E5460 with front drawer extended and E5460 components.

3.2

SOFTWARE REQUIREMENTS

The Quantum StorNext File System is a versatile, high-performance shared file system that offers heterogeneous host operating systems block-based access to the same underlying file structure housed on a single external storage system or striped across multiple external storage systems. Two of the major components of a StorNext File System environment are the metadata controller (MDC) and a client that is installed on each of the hosts that requires access to the shared file system. Communication between the MDC and the clients occurs through a segregated Ethernet network intended for use only by the StorNext File System. This private network does not necessarily have to provide large amounts of throughput, but NetApp highly recommends a very low latency between the client nodes and the MDC. The MDC is responsible for arbitrating overall access to the shared file system, as well as configuration and management. It can be configured in either a single- or dual-node arrangement for additional redundancy, and the MDC software is supplied as either Linux or Microsoft Windows based distribution. The StorNext client software is installed on each server or workstation designated for participation in the file system. Mounting and unmounting operations are largely controlled by the client software and its communication with the MDC. For additional information on the StorNext File System, refer to the Quantum Web site: www.quantum.com/ServiceandSupport/SoftwareandDocumentationDownloads/SNMS/Index.aspx.

NetApp Media Content Management Solution Based on E-Series Solutions Guide

3.3

SOLUTION ARCHITECTURE DETAILS

The E5460 is designed as a modular system that can be mounted in a standard 19-inch rack. The E5460 and StorNext package can be optimized and configured for high bandwidth and/or high read rates for a number of applications in the media markets. The E5460 is capable of sustaining up to 6GB per second reads from multiple sources and 3GB per second writes if: The writes are all sequential The reads are both sequential and random The file system impacts the performance numbers, especially for writes

Actual application performance is subject to factors of ingest, playout, sequential versus random, and configuration. Tuning of the storage and file systems is built in to the basic configurations described in this and other solution documentation. The NetApp E5460 system delivers a small footprint, efficient power usage, and an easily managed storage system. It offers high performance in one dense stack. Implementation of the StorNext File System provides high ingest and playout for media content management and other video applications. In the test case used for this solution, all of the StorNext data, including metadata, journal data, and user data, resided on a NetApp FAS2040 system. This is an optional configuration for the solution. Metadata and journals can be stored on the storage system. The E5460 in this configuration contained only media data. The StorNext metadata and file journals were stored on a FAS2040 with four 50GB LUNs. Adding the StorNext Distributed LAN Client (DLC) option to the implementation could benefit the overall design options and related data management. This solution has been developed to enable rapid file sharing between servers, and it provides cost-effective data retention. The DLC expands the capabilities of StorNext to customers who require performance increases, but without the infrastructure cost of a complete SAN implementation or the scalability limitations of other clustered NAS solutions. NETWORK ARCHITECTURE An Ethernet network is required for the management of SAN switches and storage systems (out-of-band management), and for use by the StorNext MDCs and clients. Figure 4 shows the layout of the SAN and the LAN.
Figure 4) SAN and LAN layout.

NetApp Media Content Management Solution Based on E-Series Solutions Guide

SAN SWITCH CABLING TOPOLOGY The NetApp recommended configuration for cabling is important, and special attention should be paid to the cross-connected cabling shown in Figure 5, which allows at least two paths from each server and LUN.
Figure 5) Cross-connected SAN.

TYPICAL STORAGE ARCHITECTURE The RAID type must be considered when configuring the storage arrays. NetApp recommends the following configuration as a starting point for application testing: RAID 6* or RAID 5 Volume group size of RAID 6*, 8+2, or RAID 5, 4+1 One volume (LUN) per volume group* 128k segment size* a variable starting point Cache block size: 32KB* Cache mirroring: Enabled* Media scan: Off* Cache read and write: On* Dynamic cache read prefetch: On* Storage offerings of 60 or more drives in 60-drive increments (one to multiple E5460s)

*Tested configuration and settings. TESTED STORAGE LAYOUT The E5460 system is composed of six RAID 6, 8+2 volume groups, with one volume (LUN) for each volume group that was assigned to a StorNext file system.

10

NetApp Media Content Management Solution Based on E-Series Solutions Guide

STORAGE SIZING AND PERFORMANCE CONSIDERATIONS The NetApp E5460 works well with large file I/O and small file I/O, with minor changes to accommodate the file systems and the variable and dynamic workloads of the file system used. The MCM solution is well suited for the 7200 RPM SAS drives on the E5460.

4 NETAPP MEDIA CONTENT MANAGEMENT SOLUTION

4.1 INTEGRATED SOLUTION TESTING AND OBJECTIVES

We tested the solution base components to make sure that the solution met the overall objective of providing media content management. We used typical video feeds and a playback scenario simulating a generic video archive workload. The solution met the following objectives: The E5460 system supports video ingest in a variety of formats made available through a highperformance file system. The E5460 system supports a variety of video streams (for example, HD, 2k, and 4k) in a variety of applications. The E5460 system is designed for high capacity in a small footprintup to 180GB per 4U E5460. Adding the optional Quantum StorNext Storage Manager software automatically enables policy-based data migration to lower tiers of storage while maintaining file-level accessibility, freeing up valuable space and reducing costs. The total solutions automated easy access to long-term, archived data improves timely access to large volumes of video formats.

4.2

STORNEXT FMS CONFIGURATION FILE (TEST SCENARIO)

We used the following StorNext FMS configuration file settings for our component test scenario: FsBlocksize = 64KB 256kM stripe breadth read = true write = true metadata = false journal = false userdata = true realTimeIOs = 0 realTimeIOsReserve = 0 realTimeMB = 0 realTimeMBReserve = 0 realTimeTokenTimeout = 0 multipathMethod = rotate

4.3

STORNEXT CLIENT CONFIGURATIONS

We tested the following StorNext clients: Fujitsu RX300s running Windows Enterprise Server (R2) Fujitsu RX300s running RHEL 5.5 (StorNext metadata)

11

NetApp Media Content Management Solution Based on E-Series Solutions Guide

4.4

THE E5460 TEST RESULTS

A simulated small-scale MCM solution was tested using a generic workload simulator simulating one example of a media content application. The test environment consisted of the following major components: One E5460 storage system with 60 NL-SAS disk drives Six Microsoft Windows based StorNext clients, each with two 8Gb FC ports At least one Linux based StorNext MDC

All of the StorNext user data resided on the E5460 storage systems. For the user data, the previously recommended configuration of RAID 6, 8+2 was used. A single volume composing the entire available capacity of the RAID set was used for each user data RAID set and then allocated for the StorNext File System. Metadata and journal information were configured on an external FAS2040 according to previous recommendations. Very few changes were made to the default storage system settings, with the following exceptions: Set cache block size to 32KB Disabled cache mirroring (for marginal performance improvement)

The basic configuration parameters for the StorNext File System include all of the available user data volumes in the context of a single stripe group with a stripe breadth of 256KB. Additional parameters include: status = up FsBlocksize = 64KB stripeBreadth should match RAID stripe size read = true write = true metadata = false journal = false userdata = true realTimeIOs = 0 realTimeIOsReserve = 0 realTimeMB = 0 realTimeMBReserve = 0" realTimeTokenTimeout = 0 multipathMethod = rotate>

From a benchmarking standpoint, the tools employed to simulate an MCM workload are difficult to quantify because of the vast differences in execution among the various MCM application vendors. The suite chosen for this performance example effectively performed what is believed to be a characteristic representation of a complex MCM workload, including sustained large file sequential writes as well as many smaller read streams. The primary test tool used during testing was Iometer (v. 2006.07.27). The tests were configured with the following parameters: Test case: Buffered reads/writes 100% sequential I/O Variable I/O sizes Variable read versus write percentages File sets used during testing were created to exceed controller cache to avoid caching impact

12

NetApp Media Content Management Solution Based on E-Series Solutions Guide

Table 1) Test case results: buffered reads/writes, 100% sequential, 24 total streams, E5460.

Percentage Read and Write

100% Read 0% Write 75% Read 25% Write 50% Read 50% Write 25% Read 75% Write 0% Read 100% Write

2MB I/O Size

4407MB/s 1972MB/s 1899MB/s 2131MB/s 2668MB/s

1MB I/O Size

4122MB/s 1579MB/s 1435MB/s 1571MB/s 2052MB/s

512KB I/O Size

4170MB/s 1237MB/s 1175MB/s 1200MB/s 1598MB/s

256KB I/O Size

4093MB/s 898MB/s 933MB/s 935MB/s 1227MB/s

As shown in Table 1, the tested configuration of one E5460 was able to sustain greater than 4.4GB per second of aggregate read throughput and 2.7GB per second of sustained aggregate write throughput by using buffered operations with unbounded average response times.

5 SUMMARY
The NetApp MCM solution is a scalable, high-performance, shared storage system that is optimized for storing, sharing, and serving large numbers of high-definition media files. This solution is optimal for workflows that require large, centralized storage capacity and non-real-time ingest and playout of highbandwidth media streams. It is an excellent solution for a limited number of transcoding clients, a limited number of edit stations, high-resolution media asset management, online media archiving, or centralized storage to provision to content creation workstations and to broadcast playout workstations. The combination of the NetApp E5460 and the StoreNext File System offers a wide variety of MCM solutions.

NetApp provides no representations or warranties regarding the accuracy, reliability or serviceability of any information or recommendations provided in this publication, or with respect to any results that may be obtained by the use of the information or observance of any recommendations provided herein. The information in this document is distributed AS IS, and the use of this information or the implementation of any recommendations or techniques herein is a customers responsibility and depends on the customers ability to evaluate and integrate them into the customers operational environment. This document and the information contained herein may be used solely in connection with the NetApp products discussed in this document.

2011 NetApp, Inc. All rights reserved. No portions of this document may be reproduced without prior written consent of NetApp, Inc. Specifications are subject to change without notice. NetApp, the NetApp logo, and Go further, faster; are trademarks or registered trademarks of NetApp, Inc. in the United States and/or other countries. Mac is a registered trademark of Apple Inc. Linux is a registered trademark of Linus Torvalds. Microsoft and Windows are registered trademarks of the Microsoft Corporation. Quantum and StorNext are registered trademarks of Quantum Corporation. All other brands or products are trademarks or registered trademarks of their respective holders and should be treated as such. TR-3957-0811

13

NetApp Media Content Management Solution Based on E-Series Solutions Guide