If you have video servers in your plant, you probably have not thought about how the video and audio are actually stored in the device. I am no expert on video server internals, but I know that there is a lot happening behind the scenes. And I predict big changes in the not-too-distant future.
First, let's look at the issues behind storing video and audio on disk. Incoming video is isochronous, which means that devices streaming video into a server cannot be told to wait while the server does something else. The server must always be ready to accept an incoming stream. Once it has accepted the stream, the server must continue to accept it at a continuous data rate. And video comes in at high data rates: 270Mb/s for serial digital and approximately 135Mb/s for converted NTSC. These requirements make things interesting for equipment designers.
Figure 1. Most servers store video, audio and data internally as separate files, even if they arrive at the input of the server in one file. Next-generation servers may store AAF/MXF files internally as one file. Click here to see an enlarged diagram.
Servers have to be able to handle a constant stream of high-speed data. However, all computers need time to perform various housekeeping tasks. For example, hard drives used to have to perform an occasional temperature calibration (t-cal) to be sure the heads and the spinning disk were properly aligned. During the t-cal, the hard drives would move the heads to a known physical location. Of course, if the heads were in use at the time, then the drive had to stop writing data during the t-cal. There are many other tasks computers still do regularly, including reading from the keyboard and keeping up with internal clocks. Traditional computer designers generally assume that they can stop the computer momentarily to perform these tasks without the user or application noticing. But this is not the case with a video server.
To solve this problem, engineers worked closely with disk manufacturers to change the behavior of disks themselves. They altered the microcode (the software on the disk interface itself) to make the disks more video-friendly (no t-cals, for example). They also altered the way the server operating systems worked so that they would not have to put incoming streams on hold as often to perform system tasks. Of course, it was inevitable that the servers would have to perform some system maintenance tasks. Engineers solved this problem in two ways. First, they provided dedicated processors to service the disk storage systems. That way, the processor handling disk I/O would not have to be interrupted to handle other tasks, such as reading input from the server's keyboard. Second, the engineers designed disk buffers that would allow the server to store incoming data in a “bit bucket” so it would not be lost while the system performed essential system tasks.
When servers were designed initially, disk subsystems did not have the throughput to handle video. So some designers put a matched pair of disks on a single storage card and made them look like a single disk. They alternated writing first to one disk and then the other, effectively doubling the available throughput of the disk subsystem. In some cases, they installed a completely separate data bus dedicated to moving high-speed video. The engineers behind these servers were extremely creative. They answered seemingly impossible demands and created a new class of storage device for broadcast. There is no question that video servers have forever changed the way television facilities operate.
That said, video servers were hardly off-the-shelf computers, and users paid quite a bit for the technology that made them possible. Early servers required carefully matched hardware components, optimized microcode in the hard disks and special operating system software.
Figure 2. New servers may read metadata associated with AAF/MXF files and store this data in a database. Click here to see an enlarged diagram.
Fast-forward a few years and today, as you might expect, things have changed dramatically. While video servers are still not exactly off-the-shelf items, manufacturers are certainly moving closer and closer to using off-the-shelf components in their servers. This trend will continue. You will still have to purchase a purpose-built device for storing professional quality video for some time to come, but this device will be built using commodity components and operating systems.
So, what does the future hold? As disks, processors and computer data buses become faster, manufacturers will not have to spend as much time and effort optimizing and integrating their hardware and software. But users are always demanding ever-increasing functionality, so it is possible that server vendors will have to keep crafting professional server products well into the future.
Here is an example of a new user requirement. The Advanced Authoring Format and the Material eXchange Format (AAF/MXF) used to be viewed only as an import/export model. Manufacturers have typically “hidden” the fact that they take AAF/MXF files apart and store separate video and audio files in their own formats. They then convert these at the edges. (See Figure 1) Few manufacturers expected to store information in their devices as native AAF/MXF. This is not because manufacturers do not want to use AAF/MXF, but rather because their file systems and hardware have been optimized for performance. However, as time goes on, user requirements for increased AAF/MXF functionality are growing. For instance, users are beginning to expect to be able to do the following:
The video server of the future
- Store and retrieve AAF/MXF files
- Manipulate AAF/MXF files stored on the server
- Access content stored on a video server as AAF/MXF over high-speed networks
- Archive using AAF/MXF
Manufacturers may soon find dealing AAF/MXF files natively easier than simply converting it back and forth at the edges of their devices. More powerful computers and faster networking speeds are making working with the native AAF/MXF easier. However, these new user requirements push the manufacturers away from commodity products and back into the realm of products specifically produced for the professional teleproduction market.
Users do not want to store only video and audio. They also want to store information about the video and audio. And of course, storage is only half of the job. Users want to retrieve the information as well. Metadata plays a critical part in locating and retrieving the video and audio information. Searching or retrieving information on a file-by-file basis is difficult, if not impossible. But, manufacturers are creating new video devices that enable users to easily retrieve stored content using metadata contained in the AAF/MXF wrappers. (See Figure 2.) Today's manufacturers are seeing new opportunities to develop products that will help users as they go about their everyday tasks.
Data retrieval
Brad Gilmer is the executive director of the Video Services Forum, executive director of the AAF Association, and president of Gilmer & Associates.
Send questions and comments to:brad_gilmer@primediabusiness.com
