In the realm of major facility builds, none was as challenging or as complex as the launch of the new FOX Business Network. Working within the confines of an existing space inside FOX News' main headquarters in New York City, three system integrators, hundreds of technicians and millions of dollars in equipment took the project from the initial design discussion in April 2007 to on-air in a mere six months.
When you see the all-HD facility — with its multiple control rooms, production studios, and hybrid fiber/coax infrastructure — the feat is nothing short of amazing.
FOX engineers, led by directors of engineering Doug Butler and Peter Blangiforti, acted as prime integrator and coordinated various activities of all integrators involved. The outside systems integrators on the job included (in order of involvement) Ascent Media Systems & Technology Services, National TeleConsultants (NTC), IBM and Beck Associates. All spent numerous hours and lots of manpower to convert a former parking garage and retail space into an advanced HD production plant. Ascent's team, led by Rich Bisignano, senior vice president/general manager systems integration U.S., were on-site in July with a mandate to get the new FOX Business Network operational for an Oct. 15, 2007, launch.
NTC was brought in to design and build out the broadcast IT infrastructure, HD graphics facility and end-to-end tapeless system, including the ingest and playout servers, HD editing, ingest control, associated monitoring and storage integration systems. Beck Associates designed and installed the editing environment for FOX's creative services. This included moving the entire promo, new media and online department to another floor. This environment features a large-capacity shared storage network and multiple Final Cut Pro and Avid Media Composer rooms that are responsible for both long-format and short news pieces. For master control playout, they use Omneon servers and Harris automation.
Having built the facility that supports the FOX News Channel at the same building in 1996, Ascent was familiar with the complexity of working in the 45-story skyscraper at 1211 Avenue of the Americas in NYC. A total of 40 Ascent people soon began working on the site.
Major engineering feat
Ascent's Bisignano said the project was so complex that to make it work, he had to institute the same skills and methodologies that are used on major civil construction projects. During the build, he arranged collaborative workshops in order to improve communication and identify FOX's needs upfront. These sessions are part of Ascent's “Collaborative Working” model, which helps foster productive communications between all of the various parties involved and eliminates project waste. And on a project like FOX Business News, wasting time was one luxury the teams did not have. Ascent's software integration team also provided ongoing support to FOX operations for the myriad of automation and graphics-automated events that take place virtually each minute at FOX Business News.
Multiple Thomson Grass Valley Trinix routers — including 1024 × 1024 I/O, 512 × 512 and 256 × 256 I/O configurations — manage the vast signal distribution system design. It includes 6000 cables representing more than 100mi of fiber and 100mi of copper to carry the SD and HD signals. There's another existing 512 × 512 router for SD signals alone. Much of the core routing system was pre-built in Ascent's main headquarters in Northvale, NJ. In addition, each control room has its own Evertz EQX router to drive its monitor wall.
This infrastructure also supports the FOX News Channel, which recently went to HD broadcasting as well. Both Ascent and NTC are participating in this facility upgrade (in the same building) as well.
Throughout the building, and especially for the FOX Business Network, a massive amount of Evertz conversion and distribution products, as well as its multi-image monitor wall software, have been installed. Everything is fully HD-capable, including synchronization and timing systems, signal conversion modules, three MVP multimonitor display software systems, digital rack mount displays, 130 VIP multidisplay systems, analog/digital DAs, cross- and downconverters, 200 3RU frames, audio embedders and de-embedders, and three EQX routers (216 × 288 I/O and larger) to display signals to an operator's workstation at the touch of a button.
HD production workflow
All HD production is captured in 16:9. The on-air screen includes a ticker at the bottom, with the right side wider than the left to accommodate business information on companies. Standard-definition audiences see the ticker at the bottom.
The facility now includes three identical HD control rooms, capable of displaying 238 images on each monitor wall. The displays are Tamuz monitors driven by Evertz MVP multiview software and VIP display processors. The rooms also feature one of the largest digital intercom systems in the country, made by RTS.
These spacious control rooms, with matching Sony MVS-8000 HD production switchers, support FBN and FNC production studios located on the street level. All three feature robotic pedestals with Ikegami HK-327 HD cameras. Because there are so many sources coming into the studio, there's a separate 1 M/E Sony switcher in each control room to allow a set TD to switch sources as they would a rock concert or entertainment event.
The master control area has been designed as a series of four pods, with each pod capable of controlling three HD streams. Currently, one pod is dedicated to FOX Business Network, serving as the main and back-up channels for HD and a third for SD feeds. The FOX News Channel has recently moved into this new master control area. The system is the beginning of the roadmap for the transition to HD of all FOX News shows going forward.
There's also a main ingest area, called Acquisition, where more than 125 feeds come in daily and are converted to 16:9, in both SD and HD by 10 operators. This is done in real time, so that they can be used on-air quickly.
Specially configured broadcast service panels, with audio and video connections, have been installed at strategic locations throughout the building (even the roof) and hang on the walls of the studio. They include optical-to-electrical converters for all video signals, which are required due to the long distance between the studios and the equipment centers.
An audio spider system, the Calrec Hydra, has been installed, which allows FOX to multiplex the audio signals in order to combine a large amount of sources (wireless mics, audio recorders, etc.) with ease and flexibility. The system also allows any studios' audio to be mapped to any control rooms' audio console.
Among the 10 initial edit suites for the tapeless system (now called the Digital Newsroom Systems), DV 50 compressed files are enclosed in MXF wrappers and stored on a SAN, which is accessible to all FCP edit rooms for staff collaboration. QuickTime is used as well. Of the 10 edit suites, six are intended for short-form content, and four are for long-form content. The long-form systems have their own Omneon server for edit-in-place functionality, with ingest coordinated by a media manager workstation.
File-based challenges
Moving FOX from a tape-based model (it continues to use JVC's D-9 format VTRs) to a file-based infrastructure was no easy task. This was the challenge for NTC's engineers, who worked together with IBM to deploy IT technology that could handle 16:9 SD format files, which are up-converted for HD playback. The system will eventually migrate to either DV 100 or AVC-I file format for native HD playback. Panasonic P2 cameras are currently being deployed for field acquisition.
In implementing the digital newsroom system's tapeless environment and signal-processing infrastructure, FOX decided on an IT-centric infrastructure that brings together best-of-breed solutions for acquisition, playout and automation, editing, archiving and media asset management. This includes an Ardendo asset management system, Apple's FCP for editing, Omneon servers for ingest and playout, Pebble Beach for playout automation, and IBM, which integrated the software systems and provided the robotic storage and servers.
The NTC team designed a completely redundant broadcast local area network (LAN) system, broken up into four major sections: one for the core infrastructure; one for the graphics environment; one for the digital newsroom system; and another for master control and playout. Tom Michales, who headed up the NTC team at FOX, said data throughput and security necessitated that the network be divided into several VLANs.
The NTC design supports the numerous broadcast and production systems with a dedicated LAN to accommodate the large video files. Operating in parallel with the tapeless environment on the broadcast LAN, and a smaller edit system SAN for long-form content with an Omneon server and Final Cut Pro editing systems, NTC also installed a LAN for the HD graphics network. This is running all Vizrt HD graphics systems, VDS Financial Graphics software (for the automated generation of ticker graphics) and render farms. Foundry Networks Ethernet switches were used throughout the broadcast LAN to manage the heavy amount of traffic.
Flexible IT storage environment
Also deployed are several IBM storage systems, IBM servers, Fibre Channel switches, and an IBM LTO-based tape archive system for long-term storage. There's also a substantial cache system whereby inbound feeds are recorded on Omneon servers. From there, the data files are transferred to online storage. Simultaneously, proxy files are made of the full-res material to allow Mac and PC users to view content from their desktops. Files can then be transferred to playout servers and finally sent off to the tape libraries for archiving, where they can be retrieved as needed.
NTC's Michales says this is one of the largest LAN systems in the broadcast industry, boasting a 10GB/s backbone, GigE connectivity to all devices, Fibre Channel connectivity and a control LAN with more than 2000 nodes. It's been set up to ensure total system reliability and flexibility, with a high level of resiliency and fault tolerance. It doesn't get any more sophisticated than this.
The biggest challenge for Ascent, NTC, IBM and the FOX engineers on this project was dealing with different aspect ratios and having to convert them to a common format. The system that Ascent and NTC put in place handles all signals and automatically converts them, so FOX engineers never have to worry about a piece of material's signal type.
Summary
In discussing the project with everyone involved (all of whom have worked on big jobs before), they all agree that the most unusual aspect of this job was the insane timeline. That, coupled with the fact that they were working with limited physical space (under the street level of the building) and that they had to build the facility within an operating core of the 24/7 FOX News Channel without affecting what it was doing or taking it off-air.
Many people still can't believe what they accomplished. For example, toward the end of the project, the Ascent team installed $1 million worth of equipment per day for about two weeks. The experience was positive and exhausting.
The FOX Business Network had to be on the air on Oct. 15, and the project was mostly finished by Oct. 3. Looking at the file-based production environment in place and the ease at which a massive amount of HD content is created and displayed on-air every day, now the FOX News Channel wants the same thing for its staff.
Michael Grotticelli regularly reports on professional video and broadcast technology industries.
Design team
FOX News/Business
Warren Vandeveer, sr. VP of eng. and operations
Studios, control rooms, core infrastructure
Doug Butler, dir. of eng.; Mike Young, project mgr.; Greg Ahlquest, dir. of digital media production; John Collins, Mike Liebman, Steve Blot and Warren Langrock, engineers
Graphics
Peter Blangiforti, dir. of eng.; Scott Buchholz, proj. mgr.; John Fenton, eng.
IBM Global Services Team
Bill Baer, CTO; Fabio Schiattarella, proj. mgr.;
Mike Outlaw, Brad Christus and Mike Baba, engineers
Ascent Media Services
Rich Bisignano, sr. VP/gen. mgr., systems integration U.S.; Jack Dawson, sr. VP, facilities planning; Steve Vitale, Steve Sabin: sr. proj. mgrs.; Laura Gross, dir., solutions architect group; Marilyn Pierce, dir. of IT and tech. group; Ken Brueck, lead installation supervisor; John Ciulla, sr. proj. eng.; Scott Whitcomb, broadcast LAN lead eng.
National TeleConsultants
Project management
Tom Michales, program mgr. and proj. mgr. of tapeless and long-form edit; Steven Mendel, proj. mgr. for graphics; Gary Katz, installation supervisor
Graphics
Rich Hill and William Hooper
Tapeless and long-form systems
Jim Schoedler, Bob Blanks, Don Markus, Keith DeBelius and Pete Nehl
Beck Associates
Paul Kast, dir. of eng., Northeast; Joe DiFrisco, sr. proj. mgr.
Technology at work
Apple Final Cut Pro
Ardendo asset management system
Avid Media Composer
Calrec Hydra audio spider system
Evertz
Conversion and distribution
MVP multiview software
VIP display processors
Foundry Networks Ethernet switches
Harris automation
IBM
Storage systems and servers
Fibre Channel switches
LTO-based tape archive system
Ikegami HK-327 HD cameras
Omneon servers
Pebble Beach playout and automation
RTS digital intercom
Sony MVS-8000 HD production switchers
Tamuz monitors
Thomson Grass Valley Trinix routers
VDS Financial Graphics
Vizrt HD graphics systems
