Testing Compatibility at an AES67 Plugfest

As AES67-compliant products continue to come on the market, how can we be sure they are really compatible with each other with no test equipment currently available to verify compliance?

One way is to hold a “plugfest” where manufacturers of products that incorporate AES67 can get together, hook up their products to each other and see what happens. Such a plugfest was indeed held at the beginning of November 2015 at National Public Radio headquarters in Washington, D.C. This followed up on a previous plugfest held in Munich, Germany in 2014.

“The idea of the plugfest is to be able to perform testing in a true engineering environment where ideas can be freely exchanged,” said Phil Owens, senior sales engineer, Wheatstone Corp.

Held under the aegis of the Audio Engineering Society, the plugfest was organized with the goal of checking the extent of interoperability of the 13 separate products brought in for testing. The manufacturers that participated were ALC NetworX, Archwave Technologies, Digigram, DirectOut, Lawo, Meinberg Radio Clocks, Merging Technologies, Telos Corp., Wheatstone Corp., and Yamaha Corp.

GETTING DOWN TO BUSINESS
Over the course of the plugfest a series of tests were conducted. The first test checked the ability of the devices to be synchronized to a central grandmaster clock configured for Precision Time Protocol (PTP) as defined by IEEE-1588. Actually two ports of the clock were used, each set to a different PTP domain. One of the grandmaster clock ports was set to PTP domain 0, which is the IEEE-1588 default profile; and the other for PTP domain 1, which is the media profile according to AES67, Annex A. AES67 recommends using this profile, but doesn’t mandate it.

Each clock port was connected to a different Ethernet switch, each from a different manufacturer. Then those two switches were connected together so that each PTP domain was available on both switches. In the course of testing, the various devices under test were hooked up to each of the master switches.

The next test checked multicast streaming with multiple devices transmitting streams and multiple devices in turn receiving the streams. A standard stream was used for all tests with its parameters described in Session Description Protocol (SDP).

The third test checked unicast streaming with Session Initiation Protocol (SIP). There are three SIP modes—receive only, send-only and send/receive, that were all tested.

There was also a test checking on the network itself and media clock recovery.

From left: Johan Boqvist (Swedish Radio) in discussion with Greg Shay (Telos Alliance) and Sonja Langhans (IRT). Image courtesy of AES.THE RESULTS?
How did everyone do? Pretty well, actually, although some issues were discovered. In the spirit of open discussion and discovery, the results of the tests did not identify the names of the products, which were indicated only by random letter designations.

“There are benefits for the process to have anonymity,” said Greg Shay, chief science officer, Telos Alliance. “Otherwise vendors might be hesitant to participate.”

The good news was that most of the devices were able to synchronize with the grandmaster clock in either PTP domain. One device didn’t work with one of the switches, but worked with a different non-PTP switch. And another device didn’t work with the PTP domain 1.

In the multicast streaming test, most of the devices successfully interoperated with each other, although there were some issues related to SDP. Some receivers were more sensitive than others in interpreting SDP packets. In other cases, non-compliant SDP packets were produced, which prohibited the streams from being received. Some devices could read SDP automatically, others manually. AES67 doesn’t specify how SDP data is transferred for multicast sessions.

For the unicast streaming test, most devices were interoperable in one mode or another, but there were issues with SIP. In only one caller-callee combination of devices, all three SIP modes worked. But it was more common that only the receive-only mode worked, where the caller receives the audio stream.

There were some combos where receive-only and send-only (where the caller sets up the path, but the callee receives the audio) operated correctly, and a couple of combos where only the send/receive mode (bidirectional streaming where both caller and callee are sender and receiver) operated correctly.

“We learned that there is a good level of basic connectivity between all the participants’ equipment,” Owens said. “Overall the results were promising and bode well for the continued development of AES67. Further work on SIP was identified as a good next step in refining the protocol.”

Even with the SIP issues discovered, Shay said, “implementation of session initiation protocol was less of a hurdle than some people expected.”

Besides testing interoperability among the various devices, the plugfest showed that the standard itself was well-written and unambiguous.

“The fact that the success rate was very high was taken to mean that the heart of the standard is complete, well thought-out and explained enough,” Shay said.

Another factor to the interoperability success is attributed to AES67, which—although it is a new standard—is based on technologies that have been in existence for 15–20 years, and is used in telecommunications and other fields. This made it easier to understand and implement for pro audio applications, according to Shay.

A pleasant surprise that came out of the plugfest was that one of the vendors wrote a utility tool for doing third-party translations of different methods of discovery.

“In Appendix E of AES67, there are four different methods of discovery listed, and the standard didn’t mandate the use of any one. To choose one would be limiting,” Shay said. “There is room for innovation in solving the discovery problem, and we knew it would get solved in time.” That someone brought in such a tool to help the discovery process was an example of the innovation that could be possible.

Shay said there were expectations as well as hope that manufacturers of pro audio test equipment would add an AES67 test option to their gear to aid in checking compliance.

The number of manufacturers participating in the most recent plugfest indicated to Shay that there is “wind in the sails” in implementing AES67.

“You wonder when a new standard comes out: ‘Will companies commit their resources, do their own development and make this work?’” Shay said. “We see people are leaning into this and we got a good cross-section of vendors. Without a doubt, AES67 has critical mass now. All of the vendors of proprietary audio-over-IP are on board actually delivering first workable implementations, and with this [past] fall’s AES convention and the plugfest, we feel that AES67 is over the hump and is embraced for real by all the vendors.”

Mary C. Gruszka is a systems design engineer, project manager, consultant and writer based in the New York metro are

CATEGORIES