In the past, when programs needed to be transferred from one server to another, the same process as tape-to-tape transfer often was used. It was slow (real time) and prone to errors, but because the server manufactures used proprietary compression schemes, file transfers were almost nonexistent back then. Today, file transfer and conversions are making it easier to move files from one system to another.
That being said, these different video servers don’t all use the same file formats, so file conversion is required. File conversion also is necessary when files are created in nonlinear editing (NLE) systems in a format that’s incompatible with the server. And now that spots, and sometimes full-length programs, are being transferred over IP networks, these files too need to be converted.
Before discussing how files are transferred and converted, it’s necessary to review how the various video files are constructed and the codecs used to encode the video and audio information.
The codec that is used to encode the actual audio and video information is called an essence file. The essence files for all the audio and video tracks that make up a program are held together within a container file or, as it is called in professional video, a “wrapper” file. This one wrapper file contains all the information needed to playback the selected program with all audio and video tracks as well as metadata about the essence files.
Essence files
Essence files are the basis for all audio and video files. When a video signal is encoded to a file, it might use an MPEG-2 codec, DV codec or even an M-JPEG codec — these are all essence files. They contain the data that is essential to what the viewer actually perceives. (See Figure 1.)
The type of essence file determines the quality of the audio and video signals contained within. Many video servers store their files in the MPEG format, which uses compression that results in smaller file sizes; others use DV as their native format. Whichever one is used, conversion must take place before a file of a different essence type can be played back on it.
Wrappers
Wrappers are what contain the essence files that make up a complete audio and video file or program. The wrapper holds both the audio and video essence files, in addition to metadata that contains information about what is contained inside. The metadata can consist of timing information, time code, title and segment names, air dates, kill dates and many other types of data. Different types of wrappers will carry different amounts and types of metadata. (See Figure 2.)
Conversion
When a QuickTime file (a type of wrapper) is to be transferred to a system that only accepts AVI files (another wrapper), then conversion is required. There are several issues that the conversion program must consider before carrying out the conversion. First is the type of wrapper of the original file. This must be removed along with the essence files, including all metadata information. Next, the essence files are examined. If the essence files are of the type the receiving device can work with, they are merely rewrapped in AVI and the conversion is complete. If the essence files are not compatible, then they must be converted. Changing an MPEG-2 file to DV, for instance, the MPEG-2 file must be converted into a stream of images and then compressed into the DV format at the required bit rate. This type of conversion takes time because there are many numbers to be crunched. Once the essence files are in DV format, they are wrapped in AVI and sent on their way. The metadata must also be addressed, some of which may not fit into the new wrapper’s metadata format.
Unlike the old days, today’s video servers almost all accept a range of file formats, so it is much easier to transfer files from a station’s legacy server to a new one. But the question of the metadata is still up in the air. Some broadcasters believe that all data related to a video program should be stored within the metadata of that file, but there is no real standard that specifies exactly how this should be done in a way that allows real interchange of metadata.
Video quality
Whenever an essence file is converted from one codec to another (e.g., DV to MPEG-2), the issue of video quality must be addressed. Assuming there is no data loss, the codec of the video essence is the main factor in determining video quality and, thus, its ability to be converted to another codec. The software that carries out the conversion will determine the final quality of the video and audio.
Some type of monitoring and correction of the file as it’s transferred or converted is a good idea, especially if an entire library is being moved from one system to another. This is more difficult as the transfers occur within a computer and its network and when it happens faster than real time. A few conversion programs perform such operations such as correcting gamma, changing the video’s color space and normalizing the sound volume, but they must follow a preset standard assuming that all the files contain the same errors. A QC system should be employed after all conversions are done to ensure that all the newly transferred files meet the station’s standards.
Metadata
The issue of metadata is still a hot topic long after its introduction. Although SMPTE has helped in creating the Media eXchange Format (MXF) standard and the associated DMS-1 framework, which contains space for the type of metadata that would be useful for broadcasters and video production, it is still not specific enough. There is still a great deal of wiggle room that makes it difficult for MXF files to be exchanged between facilities and to be sure that all the metadata is transferred completely. Following the introduction of MXF was the idea that all data related to a video program should be stored within the metadata of that file. On the other hand, metadata stored externally in a separate database can be searched more quickly and be added to more easily than when it’s part of the video file, which would need to be read back, recoded and then written back to storage.
How it’s done
One of the most common ways to actually transfer and/or convert a file is by using an IP network and a hot folder. Usually, a dedicated computer server has the conversion program running on it all the time. In some cases, a whole server farm is used when there are many files to be transferred in a short amount of time. On this conversion or media exchange server(s), there is a folder called a hot folder, which may actually be located on network-attached storage or similar device. The edit bays have access to this hot folder over the network, and when a file needs to be converted, the editor just moves the needed file into the hot folder. The conversion software monitors this hot folder, and when a file appears, it is automatically converted and placed in an output folder. (See Figure 3.)
Some video servers use this method to accept files directly into their storage system, and their file name is used in the video server’s database; thus, the files go directly from the NLE to the video server and are ready for on-air playout.
It is now also common for proxy files to be created when a new file is added to a video server. These proxy files are scaled-down versions of the server file in the form of MEPG-1 or QuickTime that are easy to transfer across lower-bandwidth networks (e.g., office networks) to allow viewing by managers and others.
Many file types required
Sometimes, several types of files are required for a particular station’s workflow. For instance, when a commercial spot is completed, copies for the station’s archive system and video servers are needed as well as one low-resolution version for the station’s Web site. All of these files can be made in the same way as previously mentioned. Instead of one output folder, there are several — one for each type of file. This fast, multiconversion method speeds up the workflow and ensures that everyone is looking at the same file.
Conclusion
As more stations and networks convert to an all-digital file workflow, making sure the legacy assets are converted properly and with all the required metadata is of prime importance. That starts with picking the right conversion system and training personnel to do the job correctly the first time.
