Some of you who are more experienced remember the early Ampex robotics systems of the 1970s, retrieving 2in videotape cartridges for replay of TV commercials. This was not for archive, but for online use.
By the 1990s, Ampex was gone from the broadcast video scene, and StorageTek, Sony and others were pushing even larger robotics systems for archives of videotape cassettes and data tape cartridges. With the emergence of recordable optical discs (CD-DVD), several jukebox archive systems came to market. It is fair to say, however, that none of these robotics systems encountered great success in the broadcast business, largely because of the modern video server. The business of the compressed HD storage and archive requirements for broadcasting and teleproduction will be the focus of this article — not the high-end world of uncompressed HD for digital cinema and Hollywood post.
Storage costs
A few years ago, archive playout was generally real time as SDI and manual retrieval. This was primarily because the IP digital video infrastructure was not sufficiently developed or adopted by broadcasters, though video servers were everywhere. And hard disk storage was still significantly more expensive than videocassette storage.
That relationship has changed, particularly in the fully professional segments. Sony's HDCAM tapecassette for typical camcorder use holds 42GB at a cost of $40, or less than $1 per gigabyte. A small, $40 Panasonic DVCPRO-HD tapecassette holds 25GB, which is about $1.60 per gigabyte. The latest hard disk drive with a capacity of 500GB sells for about $250, or about $0.50 per gigabyte. (See Figure 1.)
Camera storage
All emerging HD camcorders seem to have one thing in common: They don't include videotape. The internal videotape cassette drive has been replaced with internal removable flash RAM, optical disc or hard disk. Videotape is no longer a consideration when designing a future-proof archive system, except for the preservation of any old on-the-shelf cassettes.
In the IP/IT world, all connections should be electronic or networked, because physically removing and replacing storage media from one subsystem to another disrupts the design purpose and efficiency goal of file-based workflow. The only exception should be in the first step of acquisition, when nonlinear media is removed from the HD camcorder and replaced in a networked device. In most HD camcorder applications, this is necessary, as, by definition, a camcorder is used on remote locations. Wireless solutions may exist or be developed to eliminate some remove-and-replace operations (i.e. ENG); however, it is not within the scope of this article to discuss wireless in any detail.
Flash RAM cards
Flash memory cards, such as the Grass Valley CF, Ikegami RAMPak and Panasonic P2 are primarily reserved for short-term acquisition storage because they cost $60 to $120 per gigabyte. Applications are therefore seemingly limited to the remove-and-replace devices needed in HD camcorders' first step in acquisition but not anywhere else in the workflow. The old videotape cassette in the camcorder often ended up as acquisition raw storage archive on the shelf, but flash RAM cards are too expensive for such use. Therefore, the disadvantage with flash RAM is that, to preserve acquisition raw footage, an additional step is required to archive raw footage.
Optical disc cartridges
The optical disc cartridges, such as the Sony PD and InPhase holographic storage, offer one obvious advantage over videotape: nonlinear random access. The advantage over flash memory is that the optical disc can be used as acquisition raw storage archive. As a matter of fact, it may be considered deep archive, if we agree that the raw archive is likely to be accessed less in the future than the edited and formatted program derived from the raw. Sony's PD is now an established product in the company's XDCAM HD camcorders with current capacity of 23GB per cartridge, proven reliability and capacity, offering a sufficient one hour of high-quality compressed HD capacity. But what about the emerging technology of holographic optical disc recording? The InPhase Technologies Tapestry systems' optical disc cartridge are slightly larger in size than the PD, a compromise that offers the ability to hold 300GB in one cartridge.
Unfortunately, the Tapestry in its first-generation state requires a relatively large drive subsystem for the cartridge's write/read process, which adds to physical size, expense and weight and is likely to exclude HD camcorder integration for several years to come.
Sony's PD is well suited for both acquisition remove-and-replace and acquisition raw archive applications, while InPhase Tapestry may only be suitable for purpose-designed high-density deep archive applications because there is no natural process of recording (as in HD camcorder acquisition) ahead of the archive recording.
Hard disk drive
The hard disk drive arena looked pretty good five years ago. Today, at greatly elevated capacities with an economical cost of $0.50 per gigabyte, the hard disk drive is perfect for compressed HD storage at any level, including some archive applications.
The hard disk is even entering professional HD acquisition in the form of Iomega's removable REV PRO hard disk cartridge and drive. Grass Valley made it an integral part of its Infinity DMC camcorder, as did Ikegami in its HDN-X10 FieldPak. The major difference between the REV PRO and the FieldPak is that the REV PRO is a hard disk platter-only cartridge with an integral spin motor, and the FieldPak is a complete hard disk drive module.
Disc: Hard vs. optic
Both Grass Valley and Sony advocate IP and file-based workflow. And at 35GB original capacity, with Grass Valley's compressed HD bit rate somewhat higher than Sony's PD record bit rate on 23GB capacity, the REV PRO and PD are comparable. The difference is the price. Grass Valley offers an inexpensive external or internal REV PRO drive, enabling the user to ingest economically into a workstation, server or network. The PD cartridge is half the price of the REV PRO; however, Sony requires the use of a more expensive VTR-like PD cartridge player (or player/recorder) with optional GigE port providing IP connectivity to the network.
The REV PRO cartridge is certainly acquisition remove-and-replace friendly but less acquisition raw-archive friendly compared with the PD due to higher cost per hour of storage for similar quality compressed HD.
Hard disk array
The hard disk array is the most universal professional media storage technology in daily use. It has experienced exceptional growth over the past 10 years in dedicated video server systems, in SAN and NAS, and in standalone workstation add-on disk arrays.
Overall, the hard disk is difficult to beat. One inexpensive 500GB hard drive offers a sustained transfer speed of more than 50MB/s (400Mb/s) and stores more than 10 hours of high-quality compressed HD at a cost of $0.50 per gigabyte. That price, however, increases to $1 or more per gigabyte by adding array packaging, controllers and RAID.
Redundancy is something difficult, expensive or not available with cartridge storage. Think about it. A basic 10TB (10,000GB) NAS array for $10,000, with multiple GigE connections does not feature robotics, is always online, and has a true file-based workflow, with redundancy. That's the equivalent of about 300 optical or hard disk cartridges. A robotic system with a capacity of less than 100 cartridges may cost $50,000 or more.
The data tape
First as reels, then cassettes and now cartridges, data tape has been around since the early days of computers. Just like magnetic disks (hard disks), recording densities have skyrocketed, while prices have declined. One format today is LTO's third-generation Ultrium tape cartridge with 400GB of native capacity at a price of about $65 per cartridge, or $0.15 per gigabyte. Just like with Sony's PD and Grass Valley's REV PRO cartridges, archive handling of the LTO cartridge may be manual or automated through robotics.
The new archive
By definition, an archive is about long-term storage. It's a collection of records, documents or other materials of historical interest. The definition implies infrequent access.
We need to think in terms of perhaps 20 years or more. And a key factor is that the hard disk drive and the hard disk array is a completely self-contained player/recorder with permanent electronic connectivity to the network. All cartridges (and cassettes) require an external player/recorder and the mechanical process of inserting and removing the cartridge, and keeping track of the cartridge manually or by automation. It is extremely important for end users to select data industry widely adopted standards for any data tape format and to avoid proprietary formats and devices for long-term storage.
Where no access is needed for a long time, there is no need for hard disk array or robotics. Barcode and electronically inventory the data tape cartridge, place it in the vault, and (nearly) forget about it. If some access is needed in the medium term, leave the file on the nearline hard disk array. For as long as frequent access is required, the file is obviously available from the online disk array.
Storing and archiving compressed HD in an IP world means taking advantage of the great benefits of networking, where we can transfer files at any suitable speed, real time or faster or slower than real time, limited only by available bandwidth. In HD broadcasting and production, with new compression schemes like AVC-Intra and JPEG2000, a file's data rate is likely to be in the 50Mb/s to 150Mb/s range. This is a comfortable bit rate, even at multiple streams, for today's IP networking technology. We will see new and innovative approaches to facilities infrastructure over the next several years, perhaps even replacing the now entrenched HD-SDI.
Exciting times are ahead. With the current accelerated growth in program origination and in TV channels and platforms available to the population, there will still be large increases in the demand for video storage for years to come.
The most surprising conclusion of the above analysis? The demand for robotic archive systems in the future of professional HD television does not seem to be a major growth area. The primary reason? If you need a file every month or two, leave it in the nearline disk array; if you need a file every year or two (or if it's an archive master), put it manually on the archive shelf in a data tape cartridge. Because of continuing networking advances and declining costs per gigabyte, the hard disk array will continue to expand its applications and take market share away from tape-based long-term storage.
Tore B. Nordahl is principal of nordahl.tv LLC, a Los Angeles-based consulting firm delivering HDTV technology reporting, product strategy and market research services.
