HOLLYWOOD, CALIF.—Shortly before the start of the SMPTE 2015 Annual Technical Conference & Exhibition, TV Technology spoke with S. Merrill Weiss about the upcoming track of sessions “Archiving the Future and the Future of Archiving” for which he is the session chair.
TV TECHNOLOGY: What is the future of archiving? How different does it look from what we envision today as “archiving,” i.e., a sort of a virtual library full of files.
S. Merrill Weiss
S. MERRILL WEISS: Future archives are likely to be organized differently than they have been in the past. The trend already has begun toward “Object-Based” archives, as opposed to the storage of individual files as has been the practice previously. The issue with just storing files is that files that are related to one another, but of different content types, can end up in disparate locations, from which they can be difficult to retrieve when needed together. Examples of this phenomenon are files related to a single production project, which could have hundreds of thousands of image acquisition files representing individual frames, hundreds of audio acquisition files, a script in a word-processing document, a budget in a spreadsheet document, listings of props and locations in a database, binary applications to be broadcast along with the final production, and, of course, all of the files generated during post production in an array of versions. Archive Objects can collect all of the files from such a project and maintain their relationships to one another while storing them for both operational and long-term preservation purposes.
Another factor regarding future archives is that they will become much closer to what we consider to be operational storage today. There will be less differentiation between the two types of storage. Indeed, many productions today record for acquisition or post-production purposes directly to media intended for long-term storage. Doing so makes it easier to preserve all of the material created and collected as part of the production and post-production processes. The trend seems likely to continue and to accelerate.
Helping to facilitate the use of Archive Objects is a recently standardized approach developed within SMPTE called the Archive eXchange Format (AXF). AXF is a wrapper structure that enables archives to be independent of the hardware systems, operating systems, files systems and software applications on which they are created. That independence further enables archives to be recovered without the need for use of the original archive systems on which they were created. In fact, it permits recovery without the need for any archive system at all, thereby virtually eliminating the potential for orphaned archives. AXF already is seeing widespread use in the Media and Entertainment space and has the potential to extend to archiving applications of all sorts in all sorts of industries.
TVT: Will it be off-site? In the cloud?
WEISS: Future archiving is likely to be a mixture of on-site and off-site storage, with off-site and cloud storage becoming more widespread over time. Cloud storage, in particular, will depend on the use of Archive Objects to maintain relationships between files that are associated with one another as they are transferred from and recalled to producing, post-producing, and distribution locations and organizations. In fact, one of the trends in storage per se is the application of Object Storage in combination with geographic dispersal of data. Use of such techniques is facilitated by construction of Archive Objects specifically to enable streaming their data, which is the approach taken by AXF. It provides for the fastest transfer of files and their content between locations through whatever communication means happen to be applied.
TVT: With media companies creating multiple streams of content 24/7, will there have to be a systematic purge of archives?
WEISS: If anything, the trend is toward deleting less and less. With the plummeting cost of storage, it becomes more expensive to decide what is worth saving from what is not than it is just to save it all. Moreover, it is very difficult to know what will be of value in the future. With metadata bound to files and their content becoming simpler to establish and providing more relevant information about the content, appropriate matter for particular needs becomes easier to locate and recover from storage. The combination of these factors favors retention over elimination.
TVT: How important has the role of metadata become in archiving?
WEISS: As with so many aspects of media production and management, metadata has become critical in archiving. It is what allows the collection of files into Archive Objects. It is what allows the content of files to be known and selected. It holds all of the information needed for formal archival processes as needed for meaningful long-term preservation. A problem with the prevalence of metadata is that it is expensive to generate and collect. A consequence of this situation is that some content producers now are seeking to collect metadata on the set as content is created, in what have come to be called “digital birth certificates,” as a means of controlling the costs of metadata creation later. Work is under way in SMPTE to extend the AXF protocol to cover the needs of on-set creation of digital birth certificates, as it has been recognized by a number of producing organizations as a useful vehicle for providing the needed services.
S. Merrill Weiss is president of Merrill Weiss Group LLC and a consultant in electronic media technology and technology management. In a 46+ year career, he has spent more than 36 years involved in work on SMPTE standards. Merrill is a SMPTE Fellow and has received the SMPTE David Sarnoff Gold Medal and the Progress Medal. He also was a recipient of the NAB Television Engineering Achievement Award, the ATSC Bernard Lechner Outstanding Contributor Award, and the IEEE Matti S. Siukola Award.
