LTO-5 is the fifth generation of LTO tape, a tape format developed over the last decade by the LTO Program, a consortium of manufacturers consisting of HP, IBM and Quantum. Since its inception, LTO has grown to become one of the de facto tape formats installed in tape libraries by broadcast-centric organizations implementing tape- and disk-based archives. LTO-5 media can store 1.5TB of uncompressed material and write at a speed of up to 140MB/s.
What is the Linear Tape File System (LTFS)?
Is LTFS yet another file format that broadcasters should try to understand, or is it genuinely going to add value to the way we work in the industry? Perhaps the first consideration is that LTFS is truly not a file format, nor even a wrapper. It's a new way of writing data to standard, off-the-peg IT storage media. More importantly, for the first time it presents an open standard for broadcasters that can provide real interoperability between systems.
Open standard interoperability
LTFS is an open standard. Any tape written according to the LTFS specification can be read by any other system using the same specification. (See Figures 1 and 2 on page 32.) This enables true interoperability and presents several implications. Not only does this suggest the evolution of new workflows, but once-proprietary data is now opened up to be available across different vendors. This eliminates issues surrounding material viability in the event of irretrievable system damage, or even business-level, vendor-specific failures. This interoperability was demonstrated by multiple vendors at the 2011 NAB Show.
Simple transfers using standard device drivers
Those in the business of managing LTO drives can download the LTFS specification and develop their code to that standard at a base level. However, not everyone is in a position or has a requirement to deal with low-level SCSI commands to write a few media files to backup tape. One of the real advantages of LTFS is that the major drive manufacturers (HP and IBM) also provide operating system drivers that allow LTO-5 tape drives to be directly connected to PCs, Macs and Linux machines, operating at a much higher level than base-level SCSI commands.
The simplicity of this means that users (and not developers) can add a single tape drive to a small-scale system in the same way they would add a USB hard drive, for example. Simply buy the drive, download the appropriate drivers, install them and connect the drive. This then provides the user with the ability to use any operating system file management tool to move content into previously locked-down, high-capacity LTO storage. Using Windows Explorer, for example, files can be simply dragged and dropped from the server to the tape. The long-term gain for this is the ability to store large amounts of data (1.5TB per cartridge) for long periods of time without the power considerations of disk. Tracking tapes on the shelf with the content they contain becomes a consideration, but it's no more an issue than it was with videotape.
Structure of LTFS-formatted tapes
Traditionally, data tape formats have not suited the stand-alone applications as described above very well because of the distributed nature of data on the physical tape. For the first time, LTFS provides an area on tape that is specifically designed to describe the contents. This is achieved by dividing, or partitioning, the tape into two areas: the data partition for the raw data and the index partition for this descriptive data. The function of the index partition is to provide faster access to the raw data; think of it as a lookup table that points to the data.
Before the advent of this on-tape index information, any controlling system would need to scan the whole tape at read speed to extract the position of the raw data into its own index format. For an entire LTO-5 tape, this could take about three hours. However, time taken for data cataloguing is not the only issue. Once the positioning information has been extracted, it has to be stored somewhere. Traditionally, this would be a database that forms the heart of a content management system. Of course, the addition of a content management system increases the price and complexity of the system considerably. LTFS negates the requirement for this complexity.
LTFS and content management systems
LTFS clearly has a place for small-scale applications, but what about enterprise-level systems that have complex workflows, media asset management platforms and cavernous tape libraries with thousands of tapes? Perhaps the most obvious application for LTFS here is the ability to protect the assets over a long period of time. The open standard approach of LTFS means that the intellectual property required to access media can now be handed back to the content owner. This raises the question: Why would anyone need a content management system to control an archive?
The answer lies in the integration to other essential business systems (sales, billing, playout and asset management, for example). There is, however, a technical disadvantage to LTFS in these larger-scale environments, and that lies with its inability to track and identify content. Any MAM, billing or automation system uses a unique identifier (UID) and a database to track and identify individual clips and content. LTFS does not support this concept of UIDs, nor does it have the ability to a hold a UID in its native metadata structure. Therefore, the existing interfaces between these types of systems and the archive still have to be used, along with a database at the core to track assets.
Content delivery
Where LTFS really comes into its own is the ability to deliver content directly into the archive, bypassing the need to ingest from videotape, etc. Intelligent archive systems can quickly read the index information from an LTFS-formatted tape (that may, for example, have been used on location to capture material directly from HD cameras or intermediate disks) and directly assign UIDs to the content, populate the archive database and generate a low-resolution proxy (used for editing). Once the material is safely held in the archive, the management system can publish notification back to any external system to begin the synchronization process.
A master MAM system that would normally push content into the archive may use place holders for this newly arrived material. In this scenario, the filename may have to be the same as the UID (or temporary UID) from the archive management system, assigned and either adopted by the MAM system or changed in the archive to match. Either way, only minor changes to connecting APIs need to be made to adopt this approach.
An additional application for LTFS with a larger archive repository is the export of content for other external systems. It is quite likely that we're going to see the adoption of videotape-like workflows with LTO data tapes, and files replacing conventional videotape.
LTFS vs. AXF?
The interoperability of LTFS has already drawn comparison with the Archive Exchange Format (AXF), which is under design by SMPTE in collaboration with major broadcast archive vendors. It's designed to offer an archive format that can be freely written and interpreted by multiple vendors, thereby providing interoperability between previously disparate systems. The open standard nature of LTFS also allows this interoperability: Tapes written in LTFS format can be exchanged between systems and the data populated into the outlying departments. However, where AXF is a broadcast-only format, the IT nature of LTFS presents a format that operates within the broadcast arena while being accessible to nonbroadcast departments. Additionally, AXF may not become a ratified standard until later in 2012.
Conclusion
Some of the benefits of LTFS include:
- Tape content can be written to an LTFS archive group and exported to other systems;
- The exported tape, containing up to 1.5TB of data, can be transported to another LTFS-compatible system and imported, populating MAM or other systems;
- LTFS enables a single tape drive to be connected to a workstation (running Windows, Linux or Mac), with data accessible in a Windows-style drag-and-drop interface;
- LTFS provides an index partition allowing the archive to write file metadata, including positional information for rapid access.
LTFS is not simply another solution looking for a problem. There are certainly several genuine applications in the broadcast, media and entertainment industry, and as the technology is adopted, more are likely to become apparent.
Howard Twine is a product manager at SGL.
