Broadcasters have been shifting from a hybrid file-server/videotape environment to an all file-based dependency for over a decade. Fundamental to this transition is embedding the appropriate media content storage and management capabilities to meet the needs of the entire organization. Historically, audio and video recording based its operations on technologies designed to provide the bandwidth and quality of service necessary to support the demands of production. Standards-based magnetic recordings provided for media interchange, uniformity between physical hardware, and the ability to duplicate content without concern for rights, security and the like. That is all changing!
When the media industry began to embrace the IT-world, all we had for comparison were traditional business data storage and management models. We recall that videoserver technologies were initially closed loop and proprietary. Files (e.g., MJPEG) were rarely exchanged outside of the direct video-centric environment; although computer (CGI) and graphics already had a jump on file interchange. Many wrongly assumed it would be a straightforward matter to extend the structured data models of business transactions and databases into the media world, which mainly consists of unstructured data.
As the media industry moved toward file-based technologies, new expectations based upon past experience and standards were bounded by the old constraints of physical media. Industry believed that universal data storage could be leveraged to enable lower production costs and higher availability. Unfortunately, IT system providers didn't fully understand the requirements of media technologies or its industry-wide business needs. They simply took the usual practices of the PC for business applications and extended them to support media workflows and business requirements. Traditional production and broadcast users encountered difficulties as they attempted to apply their operational or business requirements to what had become an IT mass market.
Fig. 1: Digital videotape migration against file-based media storage and workflow. Using storage network infrastructures and file-based media management, either form of physical media can be processed and stored for production purposes and archive preservation. Note the transcoding processes at the far end of the workflow allows for release to digital videotape or storage to LTO-tape. This technology successfully achieves interoperability regardless of the initial format or media.
Trying to merge two differing sets of business needs and technologies into a harmonious file-based environment placed the IT and media industries into a confusion period. This "first generation" of confusion is now over, and the "next generation" demands are better understood by both worlds. With a much improved perception of how systems must work together and how they deal with fully file-based workflows, we are now able to firmly establish requirements and set metrics that can be fulfilled by equipment, software and service providers. New solutions are being applied to both enterprise wide storage systems and to the emerging cloud services market.
MEDIA DATA LIFE CYCLE
As organizations define their future they need to understand the storage issues surrounding the entire life cycle of their media data. The life cycle begins with the capture of the essence as video, audio and metadata. System architects must examine each waypoint as data forms move from acquisition to ingest, to production and packaging, and then to delivery, archive and digital preservation.
As an example, the storage requirements for digital preservation are different from short term (i.e., near line) storage or high availability (i.e., online) storage used in editorial or play-to-air services. Digital preservation and archive development is moving faster than ever; it continues to be looked at as a solution to the increasing storage capacity requirements as well as enterprise requirements for long-term preservation. Digital preservation is not unlike the previous model where digital videotape was used for both capture and mastering; with the exception that preservation is now an indirect (downstream) method as opposed to videotape, which was a direct method of storage.
Not to be overlooked are those new infrastructure requirements for networked storage. Traditionally, storage systems were closed systems set within a walled-garden of isolated VLANS, Fibre Channel switches or tightly bound proprietary architectures. With the fuse lit on 10-gigabit Ethernet and SMPTE standards already being developed for 40- and 100-GbE media transport environments, the integration of "isolated storage networks" will most likely move to become a single, homogenous environment. Switch providers are embracing the performance requirements of active media storage such as jumbo frames, large contiguous files and huge volumes of concurrent access to and from storage devices, hosts and clients. These improvements will aid in the scalability of what were once closed-network infrastructures built on generic IT components.
Finally, the emergence of cloud services for broadcast applications continues to intrigue those on the financial side as well as the technology side. This of course brings all sorts of new questions of accessibility, bandwidth, security, ownership rights and more. Private clouds, public clouds and closed (corporate) cloud storage will each play important roles going forward. While the business model for extraneous storage is somewhat understood, the model for secure collaborative cloud-based workflow is still developing. Businesses are already looking at the risks, policies and values of having their assets stored in an environment they don't fully control. With independent users already embracing the services of iCloud, Dropbox, Amazon Web Services (e.g., Cloud Drive and CloudFusion); others are wondering just how or where those solutions will fit.
The continued exploration of storage applications for media operations finds us continually uncovering new technologies that force us to change our thoughts and perceptions of how we've worked in the past. Not surprisingly, extending storage capacities and media asset management continues to be a work in progress that we can't just turn our backs on.
Karl Paulsen, CPBE, is a SMPTE Fellow and chief technologist at Diversified Systems. Read more about other storage topics in his recent book "Moving Media Storage Technologies". Contact Karl at kpaulsen@divsystems.com.
