The future of storage and how it's used is being determined today by the IT industry, where most of the components used in professional production and distribution systems and networks originate.
As the broadcast industry moves to file-based systems, storage is a key component in how facilities implement video servers into their workflow. Manufacturers have two options: use off-the-shelf IT solutions and tailor them for broadcast or design custom storage systems into their products.
In the past, the highly specialized nature of broadcast had precluded the use of off-the-shelf technology — mainly due to performance reasons. However, with the advances today in the IT industry, some manufacturers have qualified specific high-performance computer storage systems for use in broadcast. The advantage is quicker time-to-market with new technology, generally lower costs and more flexible solutions.
When looking at the storage landscape, it's important to focus on two major elements: the physical storage media (e.g. internal data processing technology and form factor) and the workflow that it will ultimately be deployed in.
The shape of space
The IT industry designs storage for several markets, with two major customers driving most of the revenue — the PC market and the enterprise market for mission critical applications. In many cases, the design guidelines for these two markets are quite different.
The PC market is primarily focused on price, while the enterprise market demands high performance and reliability. However, new technology advancements have helped to create a new category between these two, called the business critical market.
Within the enterprise drive market, several major trends are driving the use of certain types of physical storage. (See Figure 1.) The enterprise market is currently moving from Fibre Channel (FC) drives to serial attached SCSI (SAS) drives just as the PC market recently moved from parallel connections to serial SATA drives. Driving this migration was the obvious cabling advantages of a serial attachment. Serial clock speeds can be higher than parallel connections. And, SAS provides a direct point-to-point bus connection versus a shared bus connection used with FC SCSI. This means stored material can be accessed much faster. For editors and content producers, this means higher productivity.
The performance of a drive is determined by its clock speed and its rotational speed. Today, many SAS drives often operate at 15,000rpm. A drive's rotational speed helps determine how fast data can be written and then read back from the drive. In a video environment, the faster the storage system the better, but, more importantly, the data rate must be deterministic and consistent. In the IT industry, a few millisecond pause in data transfers seldom matters. However, with video, such a pause can mean black frames.
Video servers are designed with appropriate buffering to avoid these delays, but only up to a certain point. This is why a broadcast storage system must be deterministic. One area that can cause these delays is disk failures in a RAID protected system. Both FC and SAS drives provide good performance and deterministic behavior whereas SATA drives are less predictable.
The latest SATA drives — SATA 3Gb/s, running at 7500rpm — are a significant improvement over first-generation SATA 1.5Gb/s drives. In addition to clock speeds doubling to 3Gb/s, which doubled the transfer rate to 300MB/s, newer SATA drives have other video-friendly features. They include hot-swap capability, improved mean time between failure (MTBF) and Native Command Queuing. This feature enables the drives to internally optimize how commands are executed for better performance. Even so, while SATA drives continue to get better, they still lag behind enterprise SAS drives for high-performance, mission critical markets.
The business critical market
These advancements have created a new class of SATA drives, which can cost half the price of SAS drives. This becomes an important consideration for use in lower performance video applications. Seagate, which supplies a large number of drives to the broadcast and video production industry, calls these new devices business critical drives. This indicates a class of drives between enterprise SAS drives and the low-cost, lower performance PC drives.
The broadcast market is a mission critical market. While a disk problem will not result in a fatality, it can result in lost revenue. Broadcasters demand that video servers be as reliable as possible, which has resulted in the almost exclusive use of enterprise drives for professional video servers. However, new advances in these drives make them an ideal choice for other types of video production and less demanding playout applications. These SATA drives will also be available with a SAS interface. This allows engineers to pick a storage system and then populate it with either drive, based on application and budget. These new drives should begin appearing later this year.
The move from 3.5in drives to 2.5in enterprise drives will result in more compact storage systems without compromising storage capacity. This means physically smaller libraries and servers can hold more material than ever before. Broadcasters and OBs with limited space will appreciate these new 2.5in drives as they start to appear later this year.
The capacity of storage drives continues to increase at an almost unbelievable rate. Today in the PC market, there are low-performance 1TB drives, with 1.5TB drives just around the corner. The high-performance enterprise market currently uses 300GB drives, with 450GB coming later this year and 600GB drives following in the next 12-18 months. Perpendicular recording technology is allowing the disk industry to pack more bits per square inch, resulting in the need for fewer physical platters while still enjoying higher performance.
A small but growing segment of the storage industry is moving away from spinning discs to solid-state compact flash memory. The price per gigabyte is still prohibitive for most video applications, but future economies of scale will make solid-state storage an option for some video applications. Production trucks or harsh environments that value extreme robustness over massive storage will embrace this solution because solid state is not impacted by the mechanical vibrations trucks encounter.
Solid-state drives are now appearing in consumer PCs like the new Apple Air, and that trend will continue. The attraction of no moving parts, instant access and fast read time (up to 10 times faster than disks) is just too compelling to ignore.
Solid state's benefits don't come without a downside. The technology is optimal for reading, but it has limited write capability. Memory cells can only accept a limited number of writes. To counter this phenomena, solid-state disks are designed with sophisticated technology that avoids writing data to the same cells and includes additional memory that can be used as storage areas approach their limits. Spinning disks will always offer an advantage in terms of price per gigabyte, but solid state will continue to gain market share over the next few years.
It's become clear that an application will gravitate to the drive technology that best suits its needs, based mainly on performance and then price. SAS drives will certainly be considered for online editing and fast access applications. However, the less costly business critical drives are sure to make significant inroads within the production community. Their performance is certainly good enough for many applications where video I/O speed is more important than overall reliability.
Workflow efficiency
The addition of servers has helped move facilities from a baseband video infrastructure to a file-based infrastructure, bringing with it a long list of benefits that this type of IT-centric architecture affords. Broadcasters and production studios are seeing more cost savings and workflow efficiencies by migrating this way.
The whole idea of a tapeless facility goes back to implementing a workflow that gets away from real-time 270Mb/s (or 1.5Gb for HD) video and into a data file that can be as low as 8Mb/s for transfers many times faster (15Mb/s to 25Mb/s is typical for SD and 50Mb/s for HD). The most benefit is realized when the conversion (encoding) takes place as early in the production process as possible, preferably at ingest.
In addition, file-based workflow operators have the option of working with a low-resolution, browse-level version of the file (1Mb/s or less) for production, including editing, quick review or archiving. This allows engineers to cost-effectively develop internal networks where hundreds of journalists and producers can access the same file at the same time, while keeping bandwidth requirements low.
Tiered storage strategy
Another option for tapeless workflows is to implement a tiered storage strategy, with three types of storage: online, nearline and offline.
- Online: enterprise drives, highest performance, reliability and highest cost. Usually configured in a SAN system.
- Nearline: SATA drives, moderate performance, good reliability and lower cost. Usually configured in a NAS system.
- Offline: tape archive; lowest performance and lowest cost. Storage robotic systems can be as small as a desktop or as large as a bedroom.
While online and offline storage is common, the addition of nearline storage offers significant cost savings and performance improvements. For example, in a news production environment, the news director might store footage online for one week, move it to nearline for 30 days and then to offline. This reduces the amount of online storage without the large performance penalty of tape.
Infrastructure trends
The storage infrastructure is what ties everything together. Enterprise applications often rely on FC and GigE connections. FC has always been the performance leader but costs more to implement.
Gigabit technology has been predominant in the IT industry. Its performance has advanced to where it is considered a good alternative to FC in speed, yet available at a lower cost (with cheaper switches and cabling). Today, FC can support 4Gb/s applications. Ethernet is predominately a 1Gb/s platform. However, cost-effective 10Gb/s switches (actually a few 10Gb ports on a 1Gb switch) are being implemented. New technologies such as iSCSI (SCSI commands over Ethernet) and TOE cards (TCP/IP engines required for off-loading the system CPU) make GigE a good option for high-performance, deterministic video systems.
For the most common server implementations, 1Gb/s performance is certainly good enough, but when engineers need to move massive amounts of data in and out of a server, the more bandwidth the better. This is where 10Gb/s Ethernet becomes ideal. Often a high-performance FTP network will be mostly 1Gb/s with a 10Gb/s backbone — whereby most devices talk to the network via 1Gb/s but some devices have the option for 10Gb. For example, if it's necessary to move data to a very fast archive system with multiple tape drives in the 120MB/s range, it's easy to max out a 1Gb/s connection.
Embracing file wrappers
Another important aspect of achieving workflow improvements is the constant advancement of file wrappers and how video data is stored within such protocols as the Media eXchange Format (MXF), General Exchange Format (GXF) and QuickTime. The attempt to achieve a common file format that is interoperable across disparate manufacturers' platforms is gaining success. But currently there are still some ambiguities in how the standards are being implemented by each vendor. File structure, with regard to metadata, can be implemented differently.
The industry is aggressively resolving these issues, and many believe that MFX may be the best option for a truly open interoperability standard. The situation will only get better with time, because manufacturers are committed to achieving this goal.
Compression still matters
Although the price of storage continues to decline, it doesn't mean that the industry will soon be working uncompressed. In fact, the trend is just the opposite. Practical workflows will continue to rely on the use of compression in order to move files around a facility quickly and unfettered by network bandwidth. Newly emerging compression codecs like H.264 and AVC HD are improving and providing higher quality while using lower bit rates.
For example, in the 1990s, 50Mb/s MJPEG was the standard for SD broadcasts. Today 50Mb/s MPEG-2 long GOP is the standard for HD, with six times the amount of data. Technologies such as H.262 and AVC HD can cut this by 50 percent or more.
This has resulted in infrastructures that require less to deploy, while enabling users to move these smaller files around faster. This will also allow producers to distribute content outside the facility using less bandwidth.
Tomorrow
As storage demands increase — which is a certainty given the need to support multiple channels of HD content sent to a variety of distribution platforms — IT-centric technologies will continue to provide the solutions broadcasters require. Applications are still being developed, so no one's sure exactly what technology solution will be adopted.
It is clear that many vendors serving the broadcast industry have recognized the efficiencies of using off-the-shelf IT solutions in their products. Everyone benefits when open approaches are applied across an entire industry.
Roger Crooks is product marketing manager for servers and digital news production products at Thomson.
