Since its launch in 2010, LTFS has attracted growing interest from systems developers working on behalf of broadcasters and media companies. It offers security and longevity for archive storage. Perhaps more importantly, it allows designers to put the archive at the center of the asset management system, playing an active role, rather than something added at the end of the process and requiring additional steps to store and retrieve content.
This paper briefly describes LTFS, explains why it is relevant to and valuable for media organizations, and outlines some of the ways in which it can be used in new workflows.
Most will be familiar with the series of LTO archive tape formats. These were originally developed by Hewlett-Packard, IBM and Seagate (now Quantum) as an open archive standard. LTO stands for Linear Tape – Open. The three founders created the LTO Consortium, which develops the standards and makes them openly available to any vendor.
LTO is a tape format. As tape technology advances, new versions of LTO appear. The current iteration is LTO-6, although systems using earlier versions are still in widespread use.
LTFS, although it comes from the same organization, is not a tape format but a file system: Linear Tape Filing System (LTFS). Most importantly, it is a self-describing file system, using XML to define the organization of both data and metadata on each tape.
The important point to bear in mind is that it is not a wrapper, which requires each system to understand it to access the content. The tape itself is completely self-describing.
Tapes written in the LTFS format can, therefore, be used independently of any external database or storage system. An LTFS-format tape can be taken from one system and read by any other, without problems. Content can be taken from one organization’s archive and delivered to another’s asset management system without the systems having to think about how to manage the transfer. For the first time, assets are truly portable.
Some argue that it is a compromise design. The emphasis has been on the ability to take a tape from one system and read it in another. It means, for instance, that LTFS tapes cannot include permissions and ownerships: All content is accessed by the root. For media applications, which tend to have their own quite complex structure of access control, this is not a problem.
Figure 1. The strength of LTFS is its self-describing nature; an LTFS-format tape can be taken from one system and read by any other.
It is worth reminding ourselves of one other key factor at this stage. LTFS comes from an open organization within the IT industry, led by three of the leading storage suppliers, and supported by other major players. This puts the R&D resources of the IT industry behind it, which is orders of magnitude greater than anything that the media industry could bring to bear. (See Figure 1.)
In turn, this means that products are quickly optimized and commoditized, making systems based on LTFS particularly cost-effective. As future technologies develop, they will be brought into the LTFS fold, without compromising the essential portability. In years to come, LTFS content could be connected to an asset management system using any hardware from spinning disks to holographic storage, and still be instantly accessible. So LTFS represents, at the conceptual level, a long-term content preservation methodology.
LTFS in the media industry
Although LTFS is an IT standard, the LTO Consortium chose to launch the new file system at NAB in April 2010. Broadcast and media applications have been seen as important from the start.
The headline benefits include portability and extreme resilience.Because LTFS is a self-describing file system, each tape is a complete and self-contained system of assets and related metadata. A tape can be written in one archive system and will be immediately readable in another. So rapid exchange of content is hugely simplified.
There is no need for translation utilities to go from one archive or content management system to another. The index information and metadata on the LTFS tape automatically populate the database receiving the tape.
Nor do you need specialized software to write the tape. Any file management tool, running on Windows, Linux or OS X, can write to an LTFS device in exactly the way it would write to a USB stick or a removable disk drive, as shown in Figure 2.
Figure 2. The ability of LTFS devices to be written to using any file management tool running on Windows, Linux or OS X, is an advantage in delivering materials to clients.
One of the key concerns for media enterprises is the ability to maximize the value of assets by monetizing content quickly and cost-effectively. The ability to deliver materials in a standardized format, without downstream processing to create a client-specific deliverable, is a major advantage.
The same advantage of portability makes it equally well suited to disaster recovery. We talk so often about “asset management” that it can be easy to forget we really mean that content is an asset to the business, which if lost or destroyed could have a catastrophic effect on its ability to continue. Content that is archived to LTFS can be reconstructed on a completely different system, even if the original asset management is completely destroyed.So LTFS represents an excellent archive medium, which is inherently future-proof and adds an extra layer of resilience over the already highly reliable LTO format by virtue of its portability. It also has the potential of providing a simple way of delivering content, particularly where there are multiple elements rather than just a finished program.
LTFS also has the potential to transform workflows by placing the archive at the center of the operation rather than at the end.
LTFS and new workflows
Consider sports production. The typical live sports event generates a huge amount of content, and an equally huge amount of descriptive metadata from the logging team marking each play and each interesting shot.
During the event, the fast-turnaround team is creating highlights packages, which in turn become more content and metadata to be logged. But you do not know until after the event which parts of the content will be relevant or valuable. It may be some considerable time later that the significance of a player’s actions in one game become clear because of what happens in another.
The solution is to write the content, together with the logging metadata, to LTFS tape using a standalone drive, on location and as it happens. All of the material is preserved, so there is no worry about losing a vital sequence during a game or even a season.
The portability of the tapes means that they can easily be transported back to the broadcaster, the production company or the sport’s body, or indeed all three. The outside broadcast company does not need to worry about different clients needing material in different formats. Anyone can read an LTFS tape, and an intelligent archive control system can store the material instantly without the need for further processing.
That intelligent archive could also generate proxies so that the content is instantly available to all who need it. The logging data is automatically transferred to the asset management system.
The same process could also be used in reverse. A large amount of archive content could be sent to the location on LTFS tapes. Should a situation arise where it would be good to use archive footage for context, it could be recovered without the need for a dedicated asset management system, effectively by dragging and dropping from the LTFS tape.
FIMS
The Framework for Interoperative Media Systems (FIMS), the joint development from AMWA and EBU, is seen as a significant enabling technology in the move toward service-oriented architectures and largely automated workflows. It is dependent upon open systems that respond predictably to standardized messages, allowing content to be pushed around a large-scale system.
LTFS is a natural companion to FIMS. Archive systems using LTFS can sit on the FIMS network, making it part of the live content repository rather than just a back-end archive. It is a simple step to put all the enterprise’s content online at all times, without the need for offline search-and-restore processes.
Future of LTFS
To meet the challenges of more content, more deliverables and tighter budgets, the broadcast and media industry has to seize the advantages offered by off-the-shelf IT solutions. The scale of the IT market — with the consequent large R&D budgets and rapid commoditization — means that standard solutions will deliver better value, better reliability and better future evolution.
LTFS meets this model. It offers the media enterprise an architecture whereby the archive is also the primary source for content, accessible throughout the workflow.
In the future, as service-oriented architectures become more prevalent, many elements of the infrastructure will be replaced as required, rather than re-engineering the complete system. With content stored using LTFS, it will be practical to replace the asset management system as part of this ad-hoc upgrade cycle: The structure of the archive is not “owned” by a particular database.
Figure 3. Using LTFS changes the archive process by allowing tapes to be written anywhere ready for the archive and content to be easily exported.
Rather, you “own” the content, without reliance on a third-party development or application. You can take your content anywhere, confident that the essence and the metadata will be accessible by any system that has a tape drive.
Content can be written to, and read from, LTFS devices on standalone drives or large robotic libraries via multiple operating systems. Content can come and go simply by using the appropriate hardware for the task in hand. Ingest need no longer be centralized, as tapes can be written anywhere ready for the archive, as shown in Figure 3. Content can equally easily be exported for delivery to third parties or for “assets-to-go” workflows.
LTFS is being widely adopted in other industries, ensuring its success. Its adoption in the media industry will deliver simple interoperability, high resilience and a solid future-proofing strategy.
—Raj Patel is product manager at SGL.
