Figure 1. The four layers of archival storage. A near-line storage system acts as the first low-cost step in archiving.
The first asset management system I remember was a small desktop file cabinet at Ohio University Television (affectionately called the OUT House). That simple cabinet held 3in × 5in cards with a record of the entire permanent tape and film library. Archiving happened when we moved tapes out of the way to a storage room somewhere else on campus. Simplicity.
Today, there are archives that hold tens of thousands of tapes in nearby storage and many more tapes in off-site storage. Managing that many physical assets requires more sophistication than what a 3in × 5in card index could supply. Of course, the 3in × 5in card approach seemed a good deal more sensible in an era when all libraries kept a card catalog.
As low-cost computers came to the broadcast business and database programs moved from the exclusive province of mainframes to much smaller machines using off-the-shelf shrink-wrapped packages, it became practical to computerize media library records. At first, this still required a huge amount of discipline because the value of the records is entirely in knowing what you have, what it contains and where it is. The best tape librarian, computerized or not, can tell you things you could not even guess at knowing.
Digitizing the content
Archiving has been dramatically affected by computerization as well as by the infusion of video servers into our industry. Once content has been stored to a server, the bits that comprise it are no different than any other digital content. This means that complex organizations with data-intensive processes, such as those required for bank records and academic records at large universities, produce content that need not be kept beside your desk. However, it must be accessible in the event it is needed in the future.
Servers digitize analog content. The process of recording to a server makes digits out of analog content. Once there, that data can be treated in exactly the same way as bank records. There is, however, an important proviso: Content records in our industry can be many gigabytes in size and have enormous value as intrinsic assets. The most valuable diamond in the world is not likely to be as important to protect as the original footage from a motion picture by George Lucas or the assets that record the Olympics this month. This incredible rich trove of content we create in the tens of thousands of hours per day worldwide has become a staggering problem to manage and keep track of.
Clearly, the need is real for a digitized asset management system, and the computerization of our industry has made the technology approachable. However, media asset management software must store a lot more than an index number and the title of the program. The number of fields in this database can be literally dozens.
The rights information, links to the original source material, date of recording, date of last archive clone, who is in a program, episode numbers and metadata from the original shoot — including GPS and camera pointing data — are all potential elements of metadata that must be stored with, or linked inexorably to, the original content. When it is moved somewhere, the database must be updated with the location of the real asset. It might have been moved to a tape in a robotic data library, put in a box and sent off to a limestone mine, or sent to a commercial off-site data warehouse where it can be called back on a moment's notice.
The search is on
Today, many asset management systems provide rich search engines allowing content to be selected using the stored data, including visual information and caption data. Often, a link is provided that ties the original content and a proxy copy together so that desktop searches can be performed and the results viewed immediately, even when the original content has been moved to an off-site location.
It is not unreasonable to think of the proxy in a parent-child relationship to the high-res media. Proxies created for the purpose of a searchable archive of content can be quite “skinny” by using WM9, H.264 or other high-compression systems, unlike proxies made for editing, which must be editable frame by frame. The key is the link between the media, which must be absolute.
Layers
Archiving is certainly linked to asset management. However, it is important to look at archiving as more than a binary selection of locations. Today, layered storage systems are increasingly prevalent. In such a system, in addition to the online full-res media and proxies, a near-line storage system acts as the first low-cost step in the archiving process. (See Figure 1.) A near-line archive is usually built of arrays of inexpensive disks with high reliability, but lower performance than the storage for the online media content. It is often between 10 percent and 50 percent of the cost of implementing the online system.
Backing up the near-line archive is a deep archive. It may be a robotic DVD library or data tape archive with huge capacity. The cost per byte of storage is lower, and the quantity of storage available can be much larger than the combined near-line and online storage systems. The access time is much slower, with both the latency of the robotic search as well as the linear nature of the media slowing down retrieval and storage.
A fourth level of archive can be added with mirroring of the deep archive and even off-site storage of the physical media. This can further reduce the cost because the mechanical robot is not expanded linearly with increased total archive contents. The media asset management system must keep track of where all the assets are and contain expert rules to make decisions about where content should be stored. The decisions are made on expected or predicted needs or simple rules that take content older than a trigger value and move them to the next lower tier of the archive system.
Implementing a new system
Understanding the purpose is a good starting point when making decisions about implementation of asset management and archive. It is even better to thoroughly look at workflow and how such new systems can improve total operating efficiency. You might be surprised how much can be achieved without going to the most expensive system. Check into near-line storage as a transitional approach to a full implementation of asset management. You will find specialized vendors and automation companies that sell this kind of product at effective prices.
John Luff is the senior vice president of business development for AZCAR.
Send questions and comments to:john.luff@penton.com
