Ocarina for
Professional
Media
“Technologies such as
Ocarina’s file optimization
will revolutionize the
efficiency with which data is
written to disk.”
— Noemi Greyzdorf, Research
Manager, Storage Software, IDC
Digital Intermediate
Content-Aware Dedupe and Compression for Online
Storage: Media and Entertainment
The growing adoption of all-digital HD and film workflows, and the move to 3D
(stereoscopic) production has led to an explosion in capacity requirements for data file
storage. When taken in conjunction with the natural next step of moving to online archival
and content distribution and the repurposing of media assets, the overall jump in storage
demand has become a costly concern.
While the hardware costs of raw disk drive capacity is steadily declining, the overall
capital expenditures for storage nevertheless continue to rise dramatically. Operating
expenditures also are increasing along with storage growth, driven by increasing amounts
of systems administration needed for ever-larger storage systems, growing power and
cooling requirements, and the additional overhead cost of floor space to house the
physical hardware.
The Ocarina Networks solution dramatically reduces CapEx and OpEx by slashing hardware
storage-capacity requirements for digital media files. Using its unique content-aware data
reduction technologies, Ocarina can often reduce media storage costs by 50-80% or more
without impacting storage performance or workflow productivity.
This application paper provides an overview of Ocarina technology, describes a typical
media workflow, specifically citing a 2K/4K film and Digital Intermediate (DI) environment,
and identifies the savings provided by implementing an Ocarina storage optimization
solution for professional media environments.
Visit us online at www.OcarinaNetworks.com.
2 Digital Intermediate

Ocarina Networks Overview

Ocarina Networks’ storage optimization solution, the Ocarina ECOsystem, reduces the
amount of data storage capacity required in media-rich workflow environments. Utilizing
groundbreaking content-aware optimization techniques, the Ocarina solution can yield
substantial - yet still lossless - capacity reduction even with file types that are already
compressed using generic compressors.

Real-world installations show data reduction of 50-80% on initial optimization. Utilizing

Ocarina Networks groundbreaking temporal (time-based) hyper-compression techniques,
far greater data reduction levels are possible over longer-term archiving lifecycles.
Reading of the optimized data by users and applications continues unaffected while
performing at real-time speeds.

The reduction in data storage requirements enabled by Ocarina ECOsystem optimization

saves capital dollars by requiring less physical storage hardware. The commensurate
savings in systems administration, power, cooling, floor space overhead, and backup costs
can lead to substantial savings in operating expenditures as well.

Ocarina ECOsystem Architecture

The Ocarina ECOsystem utilizes a split-band deployment model where data reduction
is performed by the Ocarina Optimizer and the Ocarina ECOreader provides read
functionality when optimized data is accessed by users or applications. Optimizers are
dedicated, high-performance appliances that examine saved files based on administrator-
selected policies, apply a variety of techniques to optimize them, and writes the optimized
files back to storage. The Ocarina Optimizer is an out-of-band appliance that does not
sit in the write path or impact write performance. The Ocarina ECOreader, an in-band
solution that transparently expands files to their bit-for-bit original state, makes optimized
files available to the requesting clients or applications.

Figure 2: The ECOsystem Architecture

Ocarina Optimizer
The Ocarina Optimizer begins by identifying each file and then makes decisions about how
to optimize it for storage. This is not just generic compression or deduplication; rather it is
an intelligent set of operations determined by file content, thus the term “content-aware”.

42 Airport Parkway, San Jose, CA 95110 408.512.2966 info@ocarinanetworks.com

 Digital Intermediate 3

The overall Ocarina ECOsystem data reduction process is implemented in a three step
procedure: Extract, Correlate, and Optimize.

Utilizing its content-aware knowledge, the Extract function first unwraps and de-
compresses files (i.e. those using zip, discreet cosine transform, or other generic and
application-embedded compressors) and then delayers and decodes the file into sub-file
component objects. For example, image files will often be composed of text components
along with one or more jpg or png objects.
The overall ECOsystem
Ocarina currently supports a growing list of over five hundred different decodable (and data reduction process is
optimizable) file types. implemented in a three step
procedure: Extract, Correlate,
and Optimize.

The Extract function first unwraps

and decompresses files and then
delayers and decodes the file into
sub-file component objects.
Figure 3: Extract Phase

Once files are decomposed and

the underlying component objects
Once files are decomposed and the underlying component objects identified and identified and categorized, the
categorized, the Correlate function searches for common objects within the same file Correlate function searches for
and across other files being optimized. Unlike standard deduplication processes that common objects within the same
only recognize exact matches, the Ocarina ECOsystem Correlate function can also identify file and across other files being
similar objects, such as those images that have different resolutions or have just a small optimized.
percentage of dissimilar pixels; un-retouched and red-eye corrected pictures being a
common cross-file example. Files are then optimized, packed
into suitcases and saved.

Figure 4: Correlate Phase

42 Airport Parkway, San Jose, CA 95110 408.512.2966 info@ocarinanetworks.com

4 Digital Intermediate

The ability to correlate and deduplicate similar image objects has a large implication for
media files that often have common visual elements in sequential frames. This spatial
correlation provides significant opportunities for content-aware optimization in scenes
with well-defined animated elements that only change marginally; for example, frame-to-
frame changes – or when other parts of the images (backdrops, stationary objects, etc.) are
present in scenic frame sequences.

Thus, the Ocarina ECOsystem Correlate function works at the strata of intelligent content
information, not just raw byte-level data. This capability allows the ECOsystem to eliminate
redundant and even merely similar information both within the objects in the same files
and across objects in other files.

The last step in the process is Optimize,

wherein any of a growing catalog of
object-specific file optimization algorithms
and techniques are applied to yield the
greatest amount of data reduction.

There are currently over one hundred

different algorithmic techniques utilized by
the Ocarina Optimizer.
Figure 5: Optimize Phase

After fully optimizing each file, the

resultant optimized data is written, along
with commensurate decoding metadata, back to disk. The much larger original files may
then be automatically removed from disk providing substantial data storage reduction –
far better than any other method currently available.

Ocarina ECOreader
Optimized files are accessed by users and applications through the ECOreader, an
extremely efficient and lightweight decoder that can be utilized as software directly on
a client computer, in a dedicated gateway, or in some cases as software on the storage
system itself. ECOreader intercepts I/O requests to the original file location, retrieves the
data from Ocarina file containers where optimized data is stored, expands the file on-
the-fly, and recreates the original file bit-for-bit. This process is transparent to the user or
application—they see the files the way they expect to see them.

The Ocarina ECOreader has been designed to operate with minimal cycles and perform
similarly to the real-time performance of un-optimized files. While there is some overhead
introduced by the decoding process, this is often made up for by the reduced access times
associated with moving much smaller files off of disk and then across the network.

Any ECOreader, located anywhere, has the capability to decode an Ocarina optimized
file. Each optimized file contains all of the necessary metadata instructions for an Ocarina
ECOreader to reconstruct the original file, bit-for-bit.

Many Ocarina ECOsystem installations utilize this capability to facilitate extremely efficient

42 Airport Parkway, San Jose, CA 95110 408.512.2966 info@ocarinanetworks.com

 Digital Intermediate 5

wide-area data distribution for the purpose of content collaboration or disaster/recovery

archiving and backup. Optimization leads to reduced sizes of data transfers across
expensive and bandwidth-limited WAN links which reduces transfer times, providing
substantial savings in productivity and reducing WAN transfer costs. Film post-production has changed
dramatically over just the last few
Ocarina ECOsnap Content-Aware Snapshot years due to the market demand for
Ocarina ECOsystem technology’s unique ability to correlate sub-file level objects both ever-increasing scene complexity
within and across files has led to the breakthrough introduction of ECOsnap: time- and unique imagery. The upcoming
sequenced, hyper-compressed snapshots. industry transition to stereoscopic
3D presentation further exacerbates
Most useful in creative environments where elements, images, and media files change the challenge of delivering
daily, weekly or monthly, ECOsnap can, over time, enable unprecedented lossless profitable content on time and
compression levels of up to 200:1 while providing a unique temporal view of file changes. within budget.
Available to systems administrators directly via the Ocarina Optimizer, a built-in ECOsnap
image display engine provides the ability to rapidly page through how an image or frame
has changed over the last few days, weeks, or months.

ECOsnap, for the first time, makes keeping “long-term archive” media on immediately
accessible disk storage not only truly beneficial but also affordable.

Ocarina Networks In Digital InterMediate

Film post-production has changed dramatically over just the last few years due to the
market demand for ever-increasing scene complexity and unique imagery (with more and
more digital elements such as special effects and 3D animation). The upcoming industry
transition to stereoscopic 3D presentation further exacerbates the challenge of delivering
profitable content on time and within budget.

These changes would not have been possible without new enabling technology - such
as the introduction of high-speed film scanners, direct digital-acquisition cameras, fast
computer processors and buses capable of manipulating large images in real-time, and
high-speed networks and storage that can efficiently handle the large files and extreme
bandwidth necessary to service real-time film media.

Digital Film Production, Post Production, and Digital Intermediate

Only within the last decade has a full suite of technology become widely available that can
fully implement a digital workflow for film-resolution content. The stages can loosely be
defined as:

Acquisition and ingest; from film scanners/telecines, digital film cameras, and HD tape
machines, and elements or full frames from motion capture, animation, and special effects
rendering environments.

The individual frames and elements are stored digitally (usually on fast, real-time disk) in
the core of the Digital Intermediate post-production workflow. It is here that the scenes are
edited, images are enhanced to add or remove grain, imperfections are corrected, frames
are rotoscoped, and finally composited together into final frames, and color-corrected to
provide one or more final versions of the piece.

42 Airport Parkway, San Jose, CA 95110 408.512.2966 info@ocarinanetworks.com

6 Digital Intermediate

The Digital Intermediate (DI) creative process is generally an iterative cycle with content
often moving in and out of its high-performance primary storage environment based on
the current scene(s) being worked on. Separate, capacity-optimized storage systems are
becoming widely used for nearline storage of raw, in-use, and finished content. Tape is
often used as well; especially to take off-site for the purposes of disaster recovery and very
long-term archival. Once the productions are finalized, they may be printed on to film or
encoded digitally for distribution to theaters, or written in other formats (for DVD and Blu-
Ray versions, for example).

Disk Storage in Digital Intermediate

There are two general areas where disk
storage is incorporated with the DI
process. The first is within the creative
cycle itself, where the content is stored
for real-time ingest from various sources
and accessed by creative workstations.
The shared pool may be accessed directly
by the applications, or sometimes
local, direct-attached storage is used
on individual workstations and data is
Figure 6: Digital workflow copied to and from the shared pool by the
individual workstation.

This real-time storage is generally not extremely large. 40 terabytes of disk, for example,
would be enough for almost 40 hours of 2K footage, or approximately 10 hours of 4K
footage. For stereoscopic imagery storage, the storage requirements would double.

Real-time disk is expensive for even limited capacities due to its very high performance
characteristics, thus real-time storage is usually architected to only contain a few days of
work because of cost constraints.

Activities that require occasionally accessed content or have non-real-time requirements

are better served by much more cost effective, capacity-optimized disk systems. Often
called “online archive”, “nearline”, or “Tier-2” storage, it is common for these systems to start
at 100 terabytes of capacity or more – and grow from there.

These nearline disk systems are expanding rapidly, driven by several factors: the creative
user’s desire to have their entire library and historical versions of content within easy
access, the ease of re-purposing and re-monetizing content, as well as the efficacy of
distributing content digitally. In fact, more than a few nearline storage systems within
Animation/VFX/DI facilities are starting to reach the petabyte range, and with some
accounts, storage in the 100-petabyte range would be strongly desired by some forward-
thinking facilities.

100 petabytes of even the most cost-efficient disk storage today would be prohibitively
expensive; thus facilities are looking for a better solution.

42 Airport Parkway, San Jose, CA 95110 408.512.2966 info@ocarinanetworks.com

 Digital Intermediate 7

Utilizing Ocarina Network’s game-changing ECOsystem data reduction technology can

change the rules for large-scale media storage deployment and enable cost-effective
access to readily-accessible content for facilities wishing to expand their online archives.

Implementing Ocarina in Digital Intermediate Environments

It is possible to utilize the Ocarina ECOsystem for data stored on any disk-based storage
system. Within the DI environment; however, there is generally not enough capacity in
the high-performance disk storage tier to justify using the Ocarina solution. The nearline
storage, however, is an ideal and easily justified area to apply Ocarina technology.

If a facility is currently looking at adding capacity to its nearline disk tier, or is running out
of space on the capacity they currently have, an Ocarina solution can provide an extremely
fast return-on-investment; often at lower capital costs than adding more physical storage.
Additionally, operating expenditures are lowered as no additional systems administration,
hardware support contracts, power, cooling or floor space is required. Backup costs may
also be slashed by reducing the number of tapes needed (and the expenditure to store
The physical implementation
and maintain them).
of an Ocarina solution into the
nearline storage environment
The physical implementation of an Ocarina solution into the nearline storage environment
is a simple process.
is a simple process. One or more Ocarina Optimizers are given read/write access to the
storage, and the ECOsystem optimization process is enabled (it is also possible to read
from one storage system and write optimized files back to a different storage system). The
optimization process is fine-tuned using a policy-based engine that determines what files
should or should not be optimized. For example, excellent candidates for optimization are
the raw and finished frame images, as well as the underlying elements used in animations.

As the Ocarina Optimizer(s) processes the files, the older, un-optimized files are removed
and replaced with optimized files. If access to an optimized file is requested by a user
or application, an Ocarina ECOreader then expands the file in real-time. The Ocarina
ECOreader is a software filter driver that is implemented as a standalone Network-
Attached Storage (NAS) gateway, as software on individual workstations, or in some cases,
as software in your storage system. One or more ECOreaders may be deployed in any
combination—an installation may contain a mixture of gateways, software solutions, or
software embedded on the storage systems.

42 Airport Parkway, San Jose, CA 95110 408.512.2966 info@ocarinanetworks.com

 8 Digital Intermediate
About Ocarina Networks
Ocarina Networks is the leader
in content-aware dedupe and
compression for online storage.
Designed specifically for online
storage, the patented ECOsystem
solution gets 5–10x better data
reduction results than generic
compression or block deduplication.
Already installed at Fortune 500
companies and top websites, the
Ocarina ECOsystem easily integrates
your existing storage and processes.
We can create new free space on
your existing storage, or give your
next storage purchase 10x as much
capacity for the dollar.
Based in San Jose, CA, Ocarina is
privately held and financed by leading
investors Kleiner Perkins Caufield &
Byers and Highland Capital Partners.
For more information, visit us at
www.ocarinanetworks.com.
Assuming a minimum 50% storage savings after initial data optimization by the Ocarina
ECOsystem, any storage growth requirements that exceed 50 to 100 terabytes would be
met more cost-effectively and efficiently by the implementation of an Ocarina solution
versus acquiring, installing, maintaining, and supplying the power and cooling to new
physical storage hardware.
As capacity requirements grow, the justifications for implementing Ocarina data reduction
technology become more and more compelling, if even for only the initial data reduction
capability.
Over and above the initial data-reduction capabilities of the standard ECOsystem process,
Ocarina ECOsnap snapshots, a software option that creates time-sequenced, hyper-
compressed archives can prove truly game changing for those workflows that utilize a
significant amount of animation and special effects.
Compared to other methods of capturing changes-in-time to data that force full copies of
files to be created for even the slightest change to the file, ECOsnap only needs to record
changes at the sub-file (objects or even image pixel) level. Thus, with ECOsnap snapshots,
a scenic frame that might have subtle changes made day-by-day only requires a very small
incremental increase in disk capacity, versus the alternative of capturing a full frame of data
at every snap point when using conventional snapshot technology.
Utilizing an intuitive web-based interface with an embedded viewing engine, systems
administrators can interactively “page” – much like a “shuffle” or “flip-card” function – back
and forth through an image’s history, allowing a precise version to be recovered and
re-utilized. ECOsnap snapshots are also useful for looking back to analyze the creation
process and/or easily looking for alternative imagery.
ECOsnap technology can provide, over time and with a large number of snapshots, a truly
hyper-compression capability, with data reductions of over 200:1. Data reduction at those
levels could conceivably allow years of content to be kept online and easily accessible at
reasonable costs.
In addition to the web-based interface, standard snapshot capabilities (such as the ability
to “mount” a snapshot at a unique point-in-time) are available. A growing number of
standard file-type viewers embedded within the ECOsnap web interface continue to be
added.
Summary
Ocarina Networks data optimization products can significantly cut capital and operating
costs, reduce systems complexity, and add valuable functionality to professional
production and post-production facilities. Ocarina offerings provide compelling ROI,
with immediate storage savings realized as data is optimized, reducing the demand for
additional storage hardware versus rampant storage hardware expansion.
42 Airport Parkway, San Jose, CA 95110 408.512.2966 info@ocarinanetworks.com 04/09