Following the successful introduction of digital workflows in radio broadcasting, ARD (the consortium of German public broadcasters) decided in early 2002 to implement digital exchanging of video content via a file transfer system. A working group was established by ARD's Production and Technology Commission (PTKO) to create a video file transfer (VFT) test system, to optimize this system for practical use, and to introduce the system into daily operations at ARD's broadcast networks and major studios. DAVID Systems was chosen as the development partner for the new video transfer project.
In early 2003, the German Institute for Broadcasting Technology, also known as IRT, and the PTKO's first participating broadcasters started trial operation of the system. Further broadcasters soon followed. Due to positive results from the test system, SWR (Südwestrundfunk “Southwest Broadcasting”) had already introduced several major VFT components into regular operations in 2004, integrating the numerous tapeless workflows across its regional and state studios. At this point, SWR was still using its proprietary video formats, which were fully integrated into the system. The next step was for the station management board to coordinate the required system optimizations with IRT and DAVID Systems and to jointly implement the MXF standard. Following certification of the MXF implementation, the VFT system was introduced into ARD's regular operations in October 2007.
System concept
ARD's broadcast networks, stations and studios employ a diverse range of production and playout systems, containing a wealth of different interfaces, formats and organizational structures.
In response to this complex structural framework, the VFT was conceived as an independent system for transporting and exchanging video content. Much like a head station, the system enables broadcasters to send and receive video files via a separate system layer. (See Figure 1.) All VFT data transmission takes place within ARD's proprietary HYBNET-based intranet.
The primary component of the VFT head stations is the replicator. This component (developed by DAVID Systems) interfaces with the ARD intranet for TCP/IP file transferring and monitors the transmission quality via confirmation messages. In the case of interrupted transmissions, transfers always resume at the point of interruption. Currently, transmission via the VFT is controlled manually, but the broadcast networks are already requesting a transfer scheduling option to transport large data volumes during noncritical and low-demand time slots.
Every file released to the VFT system for transmission has to conform to the ARD file specifications. This restriction addresses the multiplicity of file formats that are currently being used, as otherwise the time and processing power required for file conversion could seriously affect VFT system performance. The receiving stations import the files directly into their production systems according to the VFT specifications. It is entirely up to each station how and via which systems the files are converted before and after the VFT transfer. The station management board refrained from instating any restrictions on the level of the individual stations.
Currently, VFT file transmission is based on a “push” concept, which means that the sender station has to trigger the transfer of the requested files. The VFT system responds by immediately transferring the transmission file (the so-called “essence”) and its metadata to the receiver station. At the receiving end, essence and metadata are stored on a file server in accordance with the sender's specifications. The receiver station's production system then fetches the data from the file server via a Web service interface for further processing.
Compression formats
ARD's many different broadcast stations employ a wide variety of compression formats. In the development of the VFT system, therefore, the PTKO decided to include several formats for compression. Sending stations may freely select from any of the specified formatting options, with preference given to the in-house format. The receiving stations, correspondingly, only need to be able to decode the specified compression formats.
The VFT system permits the following compression formats:
- MPEG 422MP/ML, type D10 (SMPTE S356);
- DV/DV50 (SMPTE S314); and
- DV (including MiniDV and DVCAM).
Where pre-agreed, formats other than these may also be used.
What are the advantages of this concept? A limited range of compression formats ensures that video quality does not deteriorate as a result of too many encoding and decoding steps. In most cases, the sending station can provide video content in its native format, i.e., at the best possible quality. This would not be possible with a system limited to just one single format. As the main priority is to provide new video items quickly and at native quality whenever possible, a range of format-related issues — such as MiniDV's unlocked audio/video signals — may need to be addressed by the receiving stations. The decision to recode the received material for further processing rests entirely with the receiving station.
Again, the underlying concept here is that VFT is exclusively responsible for the transport of the files. If any recoding is required before or after the transfer, this is performed at the discretion of the stations involved. To speed up the process, the replicator can be extended with a range of add-on functionalities to automatically recode content upon receipt, and to provide the recoded files to the processing systems in the required format.
File format and metadata
The VFT system is informed by the latest advances in format standardization and employs the MXF OP1A as its wrapping and transport format. OP1A specifies that only temporally connected portions within any given file may be transferred together. If there are sets of multiple items (items containing other items), these have to be transmitted as separate files. In ARD's program exchange system, shows are usually transmitted to the broadcast center in their entirety as single files. This method greatly simplifies the identification and cataloging of transferred items and files.
In the ARD VFT system, metadata and essences may also be transferred separately, as long as the metadata is transmitted before the essence. All metadata is transmitted using DAVID Systems' proprietary XML-based format. In principle, metadata can also be transferred directly as XML files. Replication ID numbers are assigned to uniquely associate the metadata to their corresponding essences.
As is specified in the German Television Production Guidelines, every video item's metadata contains obligatory fields such as “title,” “orderer,” “order ID,” “program type,” “source broadcaster,” technical data on audio/video, etc., and a range of optional fields. For items that are exchanged with the ARD broadcast center, the metadata must additionally contain the ARD broadcast center playback number. The VFT system automatically detects the video file's length and enters this in the mandatory field “item length.”
Metadata can be sent to the VFT system via several methods. When videotapes are exported via the system, the metadata needs to be entered manually during the ingest stage. The data entry user interface is based on the familiar tape card layout. (See Figure 2.) When video files are to be sent from other, nonlinear, systems there is a third metadata option: transfer of the XML file via a Web service. The replication process itself can similarly be triggered from a third-party system. Using a range of VFT helper applications, receiving stations can easily view and update the metadata of all video items received via the replicator.
The structural separation of metadata and essences facilitates tight integration and synchronization with the databases of nonlinear production and playout systems. If the metadata were bundled into the same file as the essence, on the other hand, the MXF file would need to be opened even for the most mundane tasks, such as forwarding the item to regional studios or content programmers.
Objectives
With the introduction of the new VFT system, ARD is pursuing several aims, such as:
- Faster exchanging of current items — reduced transfer times (faster than real time);
- Simpler workflows for exchanging items — less involvement of the station's scheduling, programming and planning staff; and
- No or minimum loss of image quality — recompression is avoided where possible.
Interestingly, the focus of these objectives shifted over the course of the project. For example, the working group's initial discussions predominantly revolved around transfer times. Other early topics included bandwidth issues, priority management for time-critical transfers, and the capability of accessing and processing files during their transfer. Indeed the VFT system has been designed to provide most of the requested capabilities, particularly faster-than-real-time transmission speeds; however, many of the original aims are now regarded as minor points by the system's users.
Initial concern about the potential degradation of video quality was dispelled through the adoption of the MXF format and the specification of the video transfer formats. The overwhelming majority of content sent out by ARD stations today is transmitted at native quality and without recompression of the in-house format. In most cases, the receiving stations do not need to perform any recoding either.
Looking back on the practical experiences from the first few months of regular operation, the project focus has shifted toward workflow optimization.
Initial practical experiences
Officially, regular operation of the VFT system started in September 2007. Many ARD stations, however, had already been using the system to exchange items and shows for many months. Today, the system has established itself as a widely used method for sending items to the ARD broadcast center. Feature-length items are also increasingly being sent via file transfer.
In the first nine months of 2007, more than 15,000 items were exchanged via VFT. This is a strong sign of user acceptance and demonstrates the system's reliability. By comparison, approximately 25,000 items were transferred via the conventional cable system during the same period. For the medium term, the PTKO expects that as the new features provided by he system become more widely known, another 10,000 to 15,000 items will be transferred via VFT rather than cable, i.e., the majority of items will be transferred as digital files.
The new system's flexibility has proven to be particularly advantageous in the course of various system migration projects. An important aspect in the conception of the VFT system was that it should not be limited to the few broadcast stations that were already based on tapeless processes at the time.
Future prospects
At ARD, the VFT system represents a major step toward fully tapeless, cross-location workflows. For stations and studios not yet equipped with nonlinear systems, VFT is a strong reason to embrace this branch of content production. Nonlinear systems will further be encouraged by the increased MXF support that ARD has demanded from production and playout system manufacturers.
Should file-based content sharing at ARD increase over the coming years as predicted by the PTKO, there will need to be ongoing investment in the technological infrastructure. Most importantly, this applies to the planned extension of the ARD HYBNET. As well as the transmission capacities needing to be extended, optimized protocols and packet sizes will need to be introduced. IRT is predicting that once the extensions and upgrades that have been planned in the short term are instated, transfer rates of up to 200Mb/s will be achieved. The highest transfer rates will be facilitated via the new UDP protocol, which is fully compatible with the replication network. The replication systems themselves will also soon need to be upgraded to provide clustering capabilities, as data volumes are rapidly increasing and availability bottlenecks are looming.
The individual broadcast stations should now focus on integrating the new functionalities into their workflows and take advantage of their potential for automation. With comprehensive VFT system integration across ARD, both the parent network and the many individual stations will benefit significantly from the cost and time reductions in the field of content exchanging.
Ingo Hahn is product management director at DAVID Systems.
