The increase in HD content for traditional broadcast, satellite and cable TV distribution, as well as the proliferation of new distribution channels for content through the Internet and mobile networks, places heavy requirements on playout systems and the digital storage that makes them possible. Pending movements to broadcast-stereoscopic content and increasingly higher-resolution content in the future will demand more bandwidth and storage capacity in order to create higher-quality viewing experiences. This article examines the drivers, requirements and projections for digital storage to support the growing professional video broadcast production and delivery market.
In late 2010, Coughlin Associates conducted a professional media and entertainment storage survey using mostly SMPTE members involved in that arena. The study was a follow-up survey to a similar one performed a year previously. This latest survey provided information guiding the recently released “2011 Digital Storage for Media and Entertainment Report,” Coughlin Associates, 2011. Data from this report is used and discussed in this article.
The full picture
As shown in Figure 1, content storage capacity for acquisition for digital TV is experiencing considerable growth due primarily to increased use of high-resolution content and eventually by 3-D TV content. Because of the performance requirements for content capture and remote requirements of field capture direct attached storage (DAS), growth is pronounced with some increase in real time of online network storage for studio and fixed work. Near-line storage is lower performance, and so its use in content capture is relatively small. Note that episodic content will demand even more storage than simple broadcast due to richer content, which tends toward the sort of storage capacity and bandwidth seen in mid-range feature films.
The physical storage media for professional cameras is undergoing rapid evolution as film and magnetic digital tape is affected by the rapid file-access convenience of hard disk drives, optical discs and, increasingly, the ruggedness of flash-based solid state storage. Figure 2 shows the percentage of various recording media used by the survey recipients in professional video cameras. While tape and HDDs are almost the same in 2009 (34 percent and 23 percent, respectively), film has dropped from 15 percent to 8 percent, while optical discs increased from 9 percent to 17 percent and flash memory increased from 19 percent to 28 percent, respectively.
Higher resolution and stereoscopic content will drive additional storage and bandwidth requirements for TV production workflows. Figure 3 shows a schematic editing station with direct attached as well as network storage.
Asked about use of direct attached and network storage in digital editing and post production, the survey gave the following statistics:
- 83.8 percent had DAS (down from 91 percent in 2009). More than 69 percent of these had more than 1TB of DAS (up to 52 percent in 2009).
- 81.2 percent had NAS or SAN (about same as in 2009). More than 58 percent had more than 16TB of NAS or SAN (up from 44 percent in 2009).
As the size of an editing facility grows, the percentage of network storage increases to facilitate collaborative workflows. Online storage also is finding use in collaborative workflows with both higher-resolution content and lower-resolution proxies.
Feeding the machine
Digital storage capacity for all active-content distribution (cable, satellite, broadcast, mobile and Internet networks) will increase about 4X from 2010 to 2016. Digital storage capacity for the distribution of content over the Internet and mobile networks is expected to increase at a faster rate, about 5X, over the same period. Video-on-demand distribution over cable, Internet and mobile networks often uses a content delivery network architecture, where frequently-accessed content is sent to cache storage locations closer to customers in order to improve their access of content and prevent overloading the central content server. (See Figure 4.)
In content distribution applications, a large near-line storage library keeps current library content, with a smaller amount of storage capacity, in a higher performance hard disk drive and increasingly solid state-drive storage for serving out content. Edge servers in content delivery networks typically contain between 1/1000 to 1/100 the storage capacity of the central server depending upon the activity in the local geographic area supported by the edge server.
Following are survey observations for video on demand electronic content distribution:
- Average hours on central content delivery systems was about 700 hours (up from 200 hours in 2009) with about 200 hours ingested monthly (this was 150 hours in 2009).
- The majority had more than five percent of content on edge servers. (Five to 20 percent was most common. In 2009, the majority of survey participants had less than 5 percent of content on edge servers.)
- About 16 percent used flash memory on their edge servers. (This was 20 percent in 2009.)
New shape coming
While current SD video broadcast has data rates of about 5Mb/s, 1080 HD content requires about 24Mb/s. 4K and 8K Super Hi-Vision (previously known as Ultra-HD) broadcast content may require 120Mb/s to 260Mb/s when it is introduced late in this decade. Significant bandwidth increases (at least 4.8X) are required to move to 4K resolution for home video display. It is much more likely that this will be available on physical distribution media (likely optical discs) before this level of resolution is available on broadcast, satellite or cable, and especially over the Internet or mobile networks. Two hours of a Super Hi-Vision movie could require more than 266GB of storage. (Compare this to 25GB on single layer Blu-ray disks and 4.7GB on DVDs.)
Providing the sort of data rates required for these high-performance content delivery systems will probably require greater use of flash memory-based high-performance storage, perhaps in front of SAS HDDs. These flash-based SSDs could be SAS, or perhaps even PCIe-based, to provide the sort of bandwidth required. Behind the content delivery storage systems, SATA drive arrays will be used for a content storage library, and magnetic tape may be used for archiving and long-term content retention.
Flash memory and other solid-state storage at the front end of content delivery systems will become more common as content resolution increases. Also, new storage and network interfaces offering higher data transfer rates will become more popular, including the new Thunderbolt direct attached interface, higher speed SAS interfaces (soon to increase to 12Gb/s), 16Gb/s fiber channel, 10Gb/s and higher Ethernet and even InfiniBand for some clustered storage applications.
Similar to increased bandwidth, front-end storage capacity to transmit HD content will grow at a similar rate. Content servers with multiple petabytes of content will be common by the middle of this decade. On the receiver side, DVRs will require several terabytes of storage for a useful number of content hours. Even streaming video content (like that provided by Netflix and Hulu) at these resolutions would require buffer caches 5X to 10X larger than those used today.
The quest to provide customers with ever-more engaging digital content will drive continued investment in capital equipment for the video production and distribution market. This will drive bandwidth and storage demand in production, transmission and consumption. Copious and lower-price distribution bandwidth, as well as more economical digital storage of all types, will bring conversion costs down to a level that encourages universal adoption.
Tom Coughlin is president of Coughlin Associates.
