LTO technology was introduced to the market more than a decade ago by the LTO Consortium, comprised of the technology provider companies (TCPs), HP, IBM and Quantum. LTO is a standardized digital tape-based storage medium, and today there are hundreds of millions in use due to its ability to provide long-term shelf-stability, high capacity, reliability and high performance. Further accelerating the adoption of LTO tape is the evolution of the media and entertainment industry, which is seeking a system for managing the dramatic increases in file-based media.
However, LTO tape alone is not ideal as an archival medium for today’s rapidly growing digital file-based content. Enter linear tape file system (LTFS), an open file system developed by the LTO Consortium specifically for LTO tape. It is designed to enable an open and more efficient way to access files on LTO tape. In fact, the LTO Consortium markets LTO with LTFS technology as having the ability to access files as a user would from a hard drive.
File systems and LTFS
In the past, hard-disk-based systems standardized on file systems, making it possible for applications to easily access files, and enabling hard drives to be exchangeable between systems. Examples include DOS and NTFS on Windows, HFS on Mac OS X and XFS on Linux. A disk file system defines two key components:
- The on-disk format for how content should be written to disk;
- The driver software that mounts the file system and makes it available for applications to read and write files.
These applications can be utility-based, like Finder and Windows Explorer, enabling the user to easily copy, move and rename files; or applications like Microsoft Word and Avid Media Composer, that directly operate on the files. These file systems allow any application to modify and access files, and have become a key way users access files on hard drives across one or more workstations. (See Figure 1.)
Figure 1. File systems allow third-party applications to directly access files on storage mediums.
LTFS is the first open and free file system for LTO tape. While hierarchical storage management (HSM) vendors provided a file system virtualization to tape before the development of LTFS, they were not free and openly accessible. Much like disk-based systems, LTFS defines an open format for how it writes content to tape. LTFS also consists of free file system driver software that allows tapes to be mounted and be seen like any other mounted device. This allows other applications to get free and open access to LTO tape. As a result, more application providers now offer LTO with LTFS systems for media workflows.
LTFS in file-based workflows
Figure 2. File-based media workflow using LTO tape with LTFS technology covers acquisition, post production, distribution and repurposing of content.
A substantial amount of content is being archived to LTO tape in pre- and post-production environments, but LTO is typically treated as an offline, nonaccessible storage platform. In contrast, the direct access nature of LTFS enables LTO tape to be used as a nearline medium. Vast quantities of file-based media can be directly accessed from LTO tape when using LTFS workflow systems. By combining nearline and archive directly on LTO tape, this storage infrastructure solves challenges and positively impacts storage costs, data management and overall efficiencies of file-based workflows from acquisition to post. (See Figure 2.)
Here are some ways LTFS can improve file-based workflows:
- Capture. With increasing resolution and frame rates, capturing raw file-based content has created data management challenges. Keeping large amounts of raw media on removable hard drives has its risks, and using NAS or SAN systems may be cost-prohibitive. Comparably, LTO tape is a low-cost, reliable storage medium. LTO with LTFS systems, when used in the field for quick camera master archiving, streamline workflows and create efficiencies for production crews. With LTFS, LTO has the potential to become the standard capture medium for raw, file-based media.
- Transport. Another challenge in file-based pipelines is the physical movement of large amounts of raw media. LTO with LTFS works well as both a system to capture raw media in the field and as a ruggedized transport format. Once delivered to post production, media on LTO with LTFS can also be used for post workflows.
- Conform. In videotape-based workflows, conforming was a predictable process, albeit slower. Media was redigitized from videotape at a higher resolution once the final edit was complete and was then relinked, finished and exported. Managing conforming with file-based media has become aa challenging task. Raw media is primarily located on removable drives, so the process of finding and relinking media is time consuming. However, by using LTO tape as a raw master format, it is possible to use LTFS-based systems to perform conforms directly from tape. The workflow uses an AAF, XML or EDL to identify the raw media across multiple tapes and uses LTFS to bring back just the correct files. Conforming from LTFS-formatted LTO tapes has the potential to make a challenging part of file-based workflows more automated, simpler and faster.
- Preview and playout. As broadcasters and other content owners create larger content archives on LTO tape, the ability to access this content becomes increasingly important. LTFS enables third-party applications to play files directly from LTO tape. The ability to play files for preview or playout for transmission gives media professionals a highly accessible archive that doubles as nearline storage.
- Transcoding and partial restore. LTFS-based systems allow third-party applications to directly access files on LTO tape to create dailies, low-resolution edit copies, distribution versions for Web upload or partial restores. Previously, this process was only possible from disk, but with LTFS, the raw master on LTO tape can also be used directly. Because multiple versions can be created from the archive master, media professionals can achieve quicker turnarounds on file-based workflows and better management of disk storage costs.
- Distribute. Remote WAN and file transfer systems (such as FTP and rsync) can directly access files from LTO tape for wide-area distribution. The ability to distribute directly from the archive medium without requiring restores to disk enables faster media workflows.
Conclusion
LTO tape is a proven storage medium for file-based assets but has largely been considered an offline medium, and accessibility has been a major drawback. With LTFS, this drawback has been eliminated. With LTFS, LTO tape can serve as both a nearline and archive, and LTFS-based systems enable file-based workflows directly from LTO tape. These systems have the ability to help film, video and broadcast professionals address the challenges of unprecedented content growth in the coming years.
—tC Chakravarty is CEO and president, StorageDNA.
