While today’s media is created and stored digitally, content production and distribution processes must now leverage high-bandwidth distribution at a level of quality equal to or better than high definition.
As a consequence, a migration is taking place across the industry that includes TV stations, film studios, production houses, and digital intermediate (DI) facilities.
High-performance intelligent storage platforms are now helping media organizations and service providers optimize workflows and improve performance. An integral component in the workflow process, these storage systems provide the scalability, performance, density, and power efficiency necessary to store and manage enormous amounts of unstructured data, which power the all-digital entertainment world.
QUANTUM SHIFT
We’ve witnessed extremely rapid growth in very large media files. Digital images, video, and audio now account for 85 percent of the global storage capacity. Most consist of unstructured data as opposed to the transaction-based data, which previously dominated many organizations. This quantum shift in the types and volumes of data has changed storage in a dramatic way.
Traditional storage architectures, once optimized for transactional data, are not well suited for digital media. Media storage platforms now handle very large files in real time. They must be able to expedite workflows and provide paths to staying ahead of the exponential growth in files and media. Systems are called upon to support hundreds of users, store millions of images, videos, and music files; and then deliver them with sub-millisecond response times.
These systems must also stream massive amounts of high -definition content without ever missing a single frame.
Storage platforms need to deliver write performance that equals read performance. They must provide a guaranteed quality-of-service (QoS) with predictable performance on large files, while simultaneously ingesting multiple sets of data streams in real and faster-than-real time.
Motion picture and digital media content creation is made possible by the collaboration of teams that make up a massive mix of production activities. Their activities include visual effects, 2D and 3D animation, full titles, trailer clips, and sound mixing.
The releases consist of feature films, television, advertising, interactive visual media and video games. In order to meet the often stringent deadlines, real-time collaborative workflow is essential. No longer can operations tolerate system downtime, inaccessible data, or reduced throughput because of data transfer bottlenecks.
Today’s workflows create content of increasing complexity that feed a distributed environment in a mix of resolutions ranging from mobile/Web resolutions through HD.
Some systems must support the 2K, 4K and 8K (UHDTV) formats of digital cinema. As a result, organizations are finding they are technically challenged when it comes to supporting a mixture of real-time workflows; concurrent multi-user editing or colorists; and growing file sizes that begin at ingest and expand throughout the post-production process.
Conventional storage systems are found to be inadequate for this level of post production, with users turning to intelligent, high-performance storage solution platforms for their answers.
Post-production environments have moved to a central storage system that supports edit-in-place functionality for dozens of uncompressed HD workstations; all the while delivering multiple streams for transmission and distribution to other services.
In extreme environments, system performance can require as much as 240 GBps of scalable storage capacity while supporting open systems with multiple CPUs, operating systems, and file systems—all in a collaborative file-sharing environment.
Broadcast entities also find themselves preparing content for multiple formats, and then distributing those multiple streams in support of consumer demands for on-air services, video on demand, streaming to mobile phones and media players, and for delivery to Web portals for Internet distribution.
These expanding activities are growing at an alarming rate, requiring another breed of high-throughput scalable storage systems that can support real-time collaborative workflows, and the storage of myriad files of unstructured images, videos, and music.
This caliber of storage system handles very large files at a very high rate of throughput. In some operations the volume of newly created content easily exceeds multiple terabytes a day, which is fed by concurrent digitization on multiple ingest stations.
These next-gen storage platforms concurrently deliver multiple terabytes of data to editing stations, playout servers, transcode and render farms, and content distribution networks. And since video playout depends on the flawless streaming of video, these storage platforms must deliver real-time performance with uncompromising reliability.
STORAGE ACCELERATORS
Heretofore, graphics and effects production had operated in a parallel job-task-related fashion using silos of individual stores attached locally to the workstation or a small NAS, or on a SAN.
By contrast, news editing has traditionally operated on a sequential basis with storage on a central SAN or NAS in support of multiple edit stations. New-era intelligent storage systems now provide for parallel storage solutions, which are changing the news and graphics/effects models through a tightly coupled centralized repository of managed files with performance equaling and usually exceeding their archaic functionality.
Storage platforms now employ storage accelerators to handle parallel paths of data between hosts and storage arrays. These accelerators utilize solid-state caches and coherency algorithms to optimize read and write operations.
By coupling scalable high-performance storage platforms to storage accelerators, the throughput and performance requirements can be better met; and can further provide for redundancy and failover paths that allow operations to continue should a host bus adaptor or switch fail.
Karl Paulsen is a technologist and consultant to the digital media and entertainment industry. Contact him at kpaulsen@divsystems.com.
