There was a time when broadcast and production equipment developed by the various manufacturers in the industry operated completely autonomously from each other’s products, frustrating users who might have to create several versions just to move content from, say, a camera to an edit system, or an edit system to a server.
Recognizing that, with the emergence of IT equipment in broadcast or production facilities something had to be done to efficiently deal with video (and audio) as data files, a group of engineers from several of these same equipment manufacturers got together in the late 1990s within the Pro-MPEG forum to develop what would become the SMPTE-approved Material eXchange Format (MXF) in 2004.
The founding MXF task force was led by Bruce Devlin, then from Snell & Wilcox, and included Oliver Morgan (from MetaGlue), Jim Wilkinson (from Sony) and Bob Edge (Thomson/Grass Valley), who were all faced with similar customer complaints: How to get a Panasonic or Sony camera to talk to a Snell standards converter, or a Grass Valley server to talk to an Avid edit workstation, and so on.
The goal of truly seamless interoperability among the various vendors was a noble one, but to this day remains somewhat elusive. Users continue to be frustrated by incompatible systems that can't read the same files between them. Some professionals take issue with the more than 30 ways that the official spec “allows” users to wrap an audio and video file (and associated metadata), including pages and pages describing 10 different ways to do the same thing.
Currently, users often have to transcode the file at each export of a device and at each import using a very low compressed format. That, according to professionals like Dan Grimes, manager of instructional production and engineering at UNLV-TV (the television production department housed within the Hank Greenspun School of Journalism and Media Studies at the University of Nevada Las Vegas), means a whole lot of time, processing cycles and wasted storage.
“When we designed our media production facility just last year, many of the manufacturers used MXF and supported common codecs within them,” Grimes said. “However, after putting the pieces together, we still ran into lots of problems when shifting the media from one program to another. We got some of the manufacturers to sit down together to come up with solutions, but not all issues could be worked out. So even if all entities say they support MXF and common codecs within them, that doesn't mean everything is going to go smoothly. We are still working to get an efficient exchange but at this time we must rely on different wrappers and codecs to exchange the essence between applications, sometimes even between programs provided by the same company.”
So, is it possible that trying to create a universal way of exchanging files is a bit too optimistic for one organization or industry to achieve?
“I don't think so,” said Devlin. “Other, much bigger, industries such as the telecommunications industry achieve better levels of standardization than the broadcast industry does. As a result, the cost of interchange is much lower in that industry, and in return operating costs can be much less. Within the broadcast industry we see a large number of chief engineers in facilities being overworked, not having enough time to follow standards and having to design solutions on the fly. These solutions don't make best use of the standards and as a result continue the ‘special cases’ syndrome that is so prevalent within the industry.”
Devlin, who now works for a Snell spin-off company called AmberFin, said even the best intentions can go awry when so many different applications are being taken into account under a single technical document.
“I have no doubt that [users ’ frustrations] aren’t an issue with the technical aspects of the specifications but is an issue with how each software code writer interprets the specification,” Devlin said. “If the specification is not clear and concise enough to convey the specifications and tolerance, it becomes inept.
“There are two issues here,” he added, “The first is the fact that MXF was designed according to two to three years of gathering requirements from the industry. It is no surprise that when the broadcast and film community insist that they need several different ways to arrange a file on disk, we end up with a specification that has several different ways of doing just that. Within the MXF specification, there are certainly areas where things could have been done more concisely, but we decided that buy-in from the community was the best result at the time.”
Most feel the confusion among professional users stems from the fact that interoperability is fundamentally a difficult problem to solve.
“In today's software environment, it is often easier for designers to ‘do their own thing’ than it is to be interoperable with a complex specification,” Devlin said. “Unless there is a drive from the user community to use open standards and use common solutions for common problems, then individual vendors will continue to build custom bespoke solutions for each ‘special case’ broadcasters.”
He said that the problem is that true compatibility requires every vendor to implement every format all the time. This in turn costs money, and those costs are passed back to the user.
“This is true, not only for MXF, but for QuickTime, Program Stream, Transport Stream, DPX, BWAV, AIFF, MP3 and all the other myriad of formats we use today,” Devlin said.
For the manufacturers that make the gear, it comes down to the way a file will be used. For example, Panasonic uses OP-Atom format (SMPTE 390M) to wrap P2 files, which the company claims is the simplest way to store audio and video clips separately and then extract them rapidly for production and editing. It also does not require processing to pull the stream (audio and video) apart, saving CPU power for other tasks.
“Although some people get frustrated by the different file wrappers, equipment vendors make choices for different reasons,” said Steve Mahrer, director of engineering for new business development at Panasonic Broadcast. “No one wrapper is better than the other; it’s just a different ‘box’ that data is placed in. Hopefully the system you are using can recognize that wrapper and unwrap it natively without having to transcode a file, which can suffer quality loss.”
He said the MXF format used is often made based on the third-party vendors a particular company is working with. For example, Avid might say it would be easier (less processing) to work with the content captured with a certain camera and stored inside a wrapper if it is parsed or separated out. This saves users time and frustration.
If you want to stream a file from a video server, a different wrapper is often used, called OP1A, because it tightly combines the audio and video into a single stream that is easier to distribute over a network. Companies like Avid, EVS, Grass Valley and Omneon use the OP1A wrapper to store and send files, as does Sony within its XDCAM recording system. In some cases, these vendors’ server technology takes an OP Adam file, unwraps it, and then rewraps it as OP1A file for better distribution and file-sharing capabilities.
For those professionals who find themselves with files that can't be easily read by their in-house video server or edit system, companies like MetaGlue, MOG Solutions and Telestream (to name a few), provide software that converts MXF wrappers from one “flavor” to another.
“Remember that when we wrap a file, we're not changing its properties, we’re only making it useable by someone other than yourself,” Mahrer said. “I think MXF has been a success from the standpoint that it gives you the ability to interchange files between different platforms. I’ve tried it, and it works. This initiative is a lot better than having to deal with a lot of proprietary formats. No one wants that anymore.”
“This is not a ‘one standard fits all’ proposition,” Mahrer added. “This is a global interchange standard with lots of options underneath it. You wrap a file according to how you intend to use or distribute it. It’s that simple.”
According to Devlin, MXF is a toolbox that does the heavy lifting of moving media around facilities. There is always more work to be done, but major broadcasters like Turner Network Broadcasting in Atlanta have done some pioneering work in the use of metadata to automate the handling and distribution of media files. Integrating this customized user metadata into an open standard without forcing vendors to redesign their products requires a little care from the standards community, but the benefits are huge for the user community.
When asked whether he thought that the MXF initiative had achieved the goals the original task force set out for it, Devlin said, not quite.
“I, personally, wanted a wrapper format that was user-driven that solved real-world problems. Unfortunately, at the launch of MXF, the format became vendor driven with the toolbox being used to provide vendor differentiation rather than a user-driven toolbox for solving real-world problems. This is changing today with organizations like AMWA defining user applications such as MXF AS02 for in-facility multiversion, multiformat production and MXF AS03 for distribution to playout servers. These user-specific constraints on MXF are likely to yield a greater level of vendor interoperability with the associated cost reduction that the user community requires.”
Forums AMWA can help broadcasters and facilities owners recognize commonality in their problems so that vendors can deliver standards-based products for common industry problems.
More information on MXF is available at www.mxf.info.
