NBC Universal's (NBCU) seismic shift away from the manual handling of digital video and audio signals began in the mid-1990s. To take the process to the next level, the broadcaster implemented an IT-centric remote control and monitoring system that allows access to thousands of individual production and distribution equipment from a single PC. This enables NBC's staff to be more productive and to support new initiatives such as digital media.
The highly sophisticated system is based on an infinitely scalable IP-based iControl system and hundreds of signal conversion cards from Miranda Technologies. The iControl architecture is based on an Element Management System design that uses telemetry probes to provide advanced facility monitoring over an in-house IP network. This replaces a system of checks and balances that included phone calls and e-mails when a system failed. It also recognizes problems and fixes them much faster than before.
The goal, according to Larry Thaler, vice president of on-air and production technology for NBCU, was to avoid system downtime and to minimize on-air disruptions as a result of failed equipment. The system also increases facility utilization, allowing faster turnaround between productions and quicker assessment of technical problems.
Monitoring across the board
Thaler and his team began deploying the SNMP-based iControl system in March, and it now monitors NBCU's main broadcast facilities and production control rooms. The system currently supervises more than 2000 devices within the network's New York City headquarters alone. The headquarters building is home to “The Today Show,” “NBC Nightly News,” “Saturday Night Live” and “Late Night with Conan O'Brien,” which share the same computer network and, in some cases, the same control rooms. The rebuilding of these shows' infrastructures was part of a company-wide migration to a serial digital infrastructure that supports both SD and HD production efficiently. Remote monitoring was always part of the plan.
The system will eventually support the entire network headquarters facility as well as all 10 NBC owned-and-operated stations and 16 Telemundo stations, giving the technical staff the ability to access and adjust the settings on specific devices in cities acrross the country, if necessary. The system can also diagnose a problem with a production switcher before it actually occurs. It monitors the network's transmission facilities, routing systems, control rooms, editing suites and new media operations.
Keeping systems operational and online
Thaler had three main goals for deploying the iControl system:
- Failure monitoringWhen an SNMP monitored device fails, a signal is sent to the technical staff via Internet connection as well as wireless pagers to alert them of a problem.
- Operations controlThe technical staff can quickly reconfigure systems to handle any type of audio and video signal. A 720p signal coming into the network's transmission facilities is automatically converted to 1080i with the push of a button. The capability also comes in handy for NBC's dubbing activities.
- New mediaThe system gives the staff a low-resolution proxy of incoming material as it's being ingested into the company's stream servers. Team members can remotely monitor these streams from anywhere NBC has a network to ensure the highest quality video images.
The system combines IP monitoring with SNMP to allow the collection of third-party equipment status and providing multivendor interoperability. This, combined with streaming media for highly visual feedback, enables staff to create highly customized graphical representations for the different departments. This makes individual device and overall system diagnosis fast and easy. A Scripted Macros feature provides automated reactions to alarm conditions and guides operators through complex diagnostics.
Getting a handle on signal attributes
One of the tricky parts of the implementation was that each one of the conversion cards the network monitors has hundreds of parameters. Multiply that by the thousands of pieces of gear in use, and you can see what a challenge it was to figure out which specific parameters were priorities and how to establish alarms for those signal attributes.
Once NBC determined which specifications it wanted to implement, Miranda helped develop a software tool that allows Thaler's team to blast that monitoring configuration out to all of the cards without affecting their signal parameters. This avoids having to reprogram every individual card in order to monitor and adjust settings remotely.
As the year rolls on, the network will continue to implement additional interfaces to new types of equipment, such as multiviewers, encoders and other types of compression equipment. Many of the vendors supplied SNMP interfaces to allow the iControl system to accurately and reliably access their respective gear. The plan is to deploy iControl software across the entire infrastructure as quickly as possible.
The network's Englewood Cliffs, NJ, facility (home to all of NBCU's cable distribution) is next on the list to deploy the software and hardware systems from Miranda, followed by the stations sometime in 2008. The NBC Olympics team will use the system in some of its transmission systems as well.
In the last two years, as NBC has converted its plants to HD, the need to monitor these systems has become more important than ever before. That's why all new HD-capable equipment must be SNMP-compatible. This allows the system to send and receive alerts via IP over a high-speed Internet connection.
Michael Grotticelli regularly reports on the professional video and broadcast technology industries.
Design team
NBC Universal
Larry Thaler, vice president, on-air production and technology
Marcus Saxton, director on-air technology
Ed Cohen, project engineer
Peter Maiorino, maintenance engineer
George Thompson, engineer
Technology at work
Evertz
MVP virtual monitor software
9725LG logo inserter
5600MSC sync generator
500ACO2 HD emergency 2×1 changeover unit
7760CCM-HD closed-captioning encoder
Miranda Technologies
iControl monitoring software
AMX, Densité, Symphonie and XVP series signal conversion modules
Kaleido-K2 multi-image display processor
Sony
MVS-8000 HD switcher
SRW-5000 HDCAM SR VTR
Thomson Grass Valley Encore and 7000 series router control systems
