Since the advent of file-based technologies, innovative and powerful implementation strategies are being used for broadcast, news, post and digital intermediate (DI) workflows. This article covers four key features of these workflows: system design considerations, process orchestration, operational aspects and workflow agility factors. Several fundamental flow models have been developed that are components in other more elaborate systems. These include high availability concepts, documenting workflow using unified modeling language (UML) diagrams, file transfer advantages, loosely coupled design ideas, service-oriented architecture (SOA) integration and more.
Key elements of media workflows
Workflows are found everywhere in life. Basically, workflow is defined as “a set of sequential steps needed to complete a job.” Some workflows are easy to implement, needing only a few tools, whereas others demand mountains of infrastructure and human capital.
Figure 1 shows the five precincts of interest for media workflows. The central box represents the actual workflow steps. The other four domains help define the “what and how” of workflow functionality. Some of these are common to all workflows, such as operational elements, whereas others, such as audiovisual streams and file-related processes, are specific to media systems.
For the media enterprise, almost agnostic to size, A/V processes are often implemented using a combination of traditional A/V plus IT gear and processes. (See Figure 2.) These hybrid systems support various workflows. Ideally, the implementation is malleable and can morph to meet the needs of future workflow changes. Also, built-in reconfigurability is vital.
Throughout this article, the constituent elements of Figure 1 will be explored with examples. The end goal is to provide a simple, high-level checklist to refer to when building a new workflow or modifying an existing one. This coverage is not exhaustive; not every aspect will be examined. However, you will be versed in the language and concepts of media flows and their design.
The design element
Any viable workflow needs a design stage. The key elements of interest are:
- Architecture
What solution space do you need? - Reliability/scale
There are 14 methods for building reliable systems. - Standards and interoperability
For example, SMPTE, IETF, IEEE, ITU-T, W3C and AMWA. - Implementation
This includes choice of vendors, systems integrator and support. - Documentation
Workflow design is not just wiring and layout.
The famed Chicago skyscraper architect Louis Sullivan said, “form follows function.” This simple yet powerful axiom applies to media workflows as well as skyscraper design. In essence, define your work space, and design it. Allow for growth, support agility and high availability as budget permits, and document not only layout and wiring but flows too. Figure 3 on page 51 shows a typical generic flow for program production. This could be modified for broadcast, live event production, news, DI, or any number of flows.
Figure 3 may be the first step in defining the work space for a new project. The level of detail is intentionally high. The design architecture will support these functions at a minimum. As the designer applies the divide and conquer rule, each process is implemented with the end goals in mind of one unified infrastructure, not islands of operations. Any given process may connect to central storage, application servers, a LAN/WAN, or traditional A/V router.
Designing for reliability
Let's consider the aspect of designing for reliability. Figure 4 on page 51 outlines 14 strategies to achieve high availability (HA) designs. For the ultimate doomsday, bulletproof system, all 14 could be applied at once. This is hardly practical, but some real-world, mission-critical systems come close. Normally, a few methods are applied, as determined by business needs.
Every design should have a service availability goal as a percentage of uptime. For example, 99.9999 percent uptime or ~32 seconds per year of downtime could be a goal. This value is achievable and allows for one or more serious failures with 32 seconds available, or less, to route around the failed element. Another approach is to decide what length of time a system can afford to be down and design from that value. Given the acceptable downtime, a designer can select options from Figure 4 to meet this goal.
Important concepts in HA designs are a single point of failure (SPOF) and no single point of failure (NSPOF). An SPOF device is not designed with redundant components. If any critical component fails — such as power supply, controller or storage — then the unit often fails. SPOF devices are used when NSPOF is not necessary or affordable. NSPOF, on the other hand, is used when the element is a critical link in a workflow. For example, most large video server storage arrays are designed with NSPOF thinking. In this case, any single element can fail, and the unit keeps plugging along with no apparent downtime. NSPOF devices have dual power supplies, multiple fans, duplicate network I/O, dual controllers, passive backplanes and more.
Some themes in Figure 4 are dual elements and I/O, dual pathing, NSPOF, mirrors, self-healing, caching, and failover to a spare. Each of these helps to resolve a particular device, link, I/O port, store or control point failure. Auto failover is vital when coupled with other methods. When a standalone element fails, such as a link, it is necessary to reroute the data traffic to an alternate link automatically. So, early fault detection is also paramount in an auto failover scenario. The total time it takes to detect and correct, or bypass, a fault is important.
Standards ubiquity
No practical system should be constructed without applying standards from a broad range of sources. Gone are the days when SMPTE standards were the only glue needed to create a video system. Today, in addition, we need the standards from the Internet Engineering Task Force (IETF), W3C and the Institute of Electrical and Electronics Engineers (IEEE) among others. User groups such as the Advanced Media Workflow Association have a mission to create best practices and recommended technologies for implementing workflows in networked media environments. They are currently developing specifications for the Advanced Authoring Format (AAF) and Material eXchange Format (MXF) component inventory method (AS-02) and a constrained MXF version for program delivery (AS-03) to broadcasters and other specifications.
Workflow documentation methods
Finally, under the design banner, consider documentation. For many years, facility documentation was a collection of diagrams showing equipment layout and racking with detailed wiring and inventory diagrams. This level of documentation is still necessary but not sufficient to describe the workflow component of an installation. While not every installation will require a workflow diagram, many will. If the workflow is complex, with disparate processes and time-related dependencies, the following methods should be of value.
Workflow stakeholders include the analysts who create and refine the processes, the technical developers responsible for implementing the processes, and the technical managers who monitor and manage the processes. Two diagramming methods are gaining acceptance to define process flow. One is based on Unified Modeling Language (UML) and another on Business Process Modeling Notation (BPMN).
UML offers activity, sequence, communication and timing diagrams. Which one should be used to describe a media workflow? Well, each has a special purpose, so depending on what aspects are to be communicated, the choice is flexible. The activity diagram offers the most flexible layout for media workflow, but all will find usage.
Another graphical modeling tool is BPMN. Don't be put off by the business term in the name. The tools translate to media workflows as proven by their usage by some systems integrators. The modeling in BPMN is made by simple diagrams with a small set of graphical elements. The four basic categories of elements enable designers to describe media flows:
- Flow objects: events, activities and gateways.
- Connecting objects: sequence flow, message flow and association.
- Swim-lanes: pool and lane.
- Artifacts: data objects, group and annotation.
Fortunately, there are reasonably inexpensive layout tools for both UML and BPMN.
Process orchestration
There are three primary media flows using networked media techniques. (See Figure 5.) Physical videotape methods are not included.
Digital networking has enabled these three modes of media transfer. Mode 1 is the ubiquitous file transfer at rates less than, approximately equal to and greater than video real time. The second mode is streaming over IP or some other link type. The mother of this type is video over SDI — the workhorse of the modern facility. Video over SDI is not considered networked media in that SDI is not networkable but exists as a switched circuit.
Video over IP is common for distribution to the end user, including Web media and IPTV related services. In the professional facility, video over IP has yet to replace SDI. Until Ethernet has an equivalent synchronous version with guaranteed low latency (QoS specs like SDI), SDI will remain the ruling incumbent. The IEEE is developing such technology. The 802.1 Audio/Video Bridging Task Group is developing a comprehensive set of standards to enable high-quality, low-latency streaming of time-sensitive applications. These standards will specify a means for time synchronization (IEEE 802.IAS), a resource reservation protocol (IEEE 802.1Qat), and a set of forwarding and queuing rules that bound the variability of delay in an AVB network (IEEE 802.1Qav). These are new standards, and it will require time for the industry to embrace them if it ever does.
The third method of media transfer is storage access. This is distinctly different from the other two methods. Storage access supports the read/write of data in random access style using storage area network (SAN) or network attached storage (NAS). There are storage systems specifically designed to support the real-time (in a video sense) access to hundreds of simultaneous Ethernet-connected HD media clients with no-excuse data delivery.
These three methods are the building blocks for the modern media facility. Designers must use wisdom when selecting one method over another. A big mistake when doing a new design is to mimic a videotape workflow using networked media. Videotape flows are limited in many ways, and networked media allows for many dimensions not permitted using only tape.
Comparing flow types
Next, let's compare three flow types — one using pure streaming and two using file transfer. (See Figure 6 above.) The general idea is to process an incoming video signal program as follows: ingest/record, apply an aspect ratio convert and add side panels, add a lower-third graphic and finally output the result. The top flow is most commonly seen using SDI (or AES/EBU links for audio-related applications) connectivity. Of course, a process may be any transform or human-assisted means. For live events demanding a minimal in/out latency (few video frames), SDI streaming connectivity is often used.
The middle flow uses the bucket brigade method. First, the entire program is ingested and saved to storage. Then, either by a file transfer between processes or via an intermediate “parking place” storage device, the program file is moved to the aspect ratio converter (ARC) process then to the graphic composite overlay process, and finally is output. In each step, the entire file is imported to a process and then exported to the next process in the chain. Hence, the bucket brigade nickname.
The delay from in to out can be quite large, on the order of 10 minutes to 20 minutes (total ARC and composite process delay) for a 60-minute program, not counting the time to ingest the incoming program (60 minutes). The faster each individual process, the shorter the total in/out delay. A designer would choose this method when completion delay time is not a critical metric. For example, if the completed program is needed the following day, then by all means, use this flow method. When timing is not critical, low-speed transfers and slow processes may be used. Note, too, that individual processes do not need to work in real time, and this relaxes their design. In fact, most A/V processes can work faster than real-time video.
The third flow is useful when the in/out delay needs to be low delay, but not as short as the fully streamed flow. Basically, the target file is continuously streamed between processes. This style may be called “process-while-record,” with “edit-while-record” being a common version. So, as process 1 finishes say the first 30 seconds of its task, process 2 can start its task and so on. There's no need to wait for an entire process to complete before another process in the chain can start, as with the bucket brigade. As long as the next process does not work ahead of the succeeding process, then all is well. Process timing is critical. Edit-while-record is used with some breaking news story production. The editor can start the task even while the program material is being recorded, with consequent faster time to air.
These flow methods may be combined in hybrid means to form serial/parallel versions. Decision steps are also needed, so a process branch may occur. A rules engine may be useful to automatic process tasks. Look for opportunities to automate common process chains. This does not only save manpower and speed up a process chain, but also it yields metrics to improve the flow.
The metric of money
One common metric of interest to all facility managers is daily operational cost of a given process flow. If file transfer is involved between site locations, the cost of bandwidth may be a large factor in workflow cost. We all know the proverb time equals money. In the spirit of the analogy, there is a corollary to this: 1/Time = Money. This is also true because 1/T is rate (such as Mb/s), and we pay for transfer date rates. A 1Mb/s WAN link costs substantially less than a 100Mb/s link.
The operational element
When defining a workflow process step, it is good practice to define each of the following five characteristics for each step. For this example, let's use the create proxy step in Figure 3.
- What are the specs for the create proxy step (encoder type, data rates, speed, file format, invoke means, input/output means, etc.)?
- How will create proxy be implemented? Use Vendor X's encoder, running on Vendor Y's server connected to storage main (reliability means, failover means, scale means, monitor means, Web services means, API means, etc.).
- When will create proxy be invoked, (workload, duty cycle, peak/average jobs, etc.)?
- Where will create proxy be located (local instantiation, remote service, contracted service, etc.)?
- Who will own and manage the create proxy service (A/V operations, IT operations, contracted operations, etc.); and who will use it (number of invokers, department, etc.)?
The process of answering each question forces a designer to think deeply about each process. This way, hidden efficiencies may be uncovered, such as we can share server Y with other processes, because the create proxy workload is small even with two times more loading. Or, we can locate server Y in room M, because there is ample power and cooling available.
Performance QoS, reservations, exceptions
Related to the five questions discussed previously, it is valuable to define the performance quality of service (QoS), reservation methods and error handling as separate aspects. Documenting application QoS is useful when scaling or changing a service in the future. When a service is shared (edit suite), then a reservation system with time, billing and resource usage may be required. Providing a system-wide reservation board (similar to a flight departure/arrival schedule display) available for all to see is often good practice at a large facility.
Exception handling deals with hardware and software warnings, faults, errors and status messaging. When something is out of order, it should be registered and personnel notified to repair it. Exceptions may range from warnings of intermittent errors, to out-of-spec signals (audio peaks), to resource status (90 percent full), to a complete device failure. High-end infrastructure monitoring systems are available from traditional IT vendors. A/V specific solutions are available from several vendors.
Investing in monitoring and diagnostics products is a matter of return on investment for the most part. If you are offering services, such as creative tools by-the-hour or commercial program playout, then money spent on keeping the facility up and running is worth the investment.
Workflow agility
Agility is defined as the ability to move in a quick and easy fashion, or change directions quickly. Typically, media workflows are purpose-built — broadcast, post, animation, news, live events, DI and so on. It is prudent, as discussed, for form to follow function. Nonetheless, within the bounds of scale, future changes and reconfigurations, a workflow should be agile.
Loosely coupled designs
One key aspect of flexibility is the concept of loose versus tight coupled processes. This industry is built on tightly coupled designs — hard-wired, rigid, A/V systems with little flexibility for reordering, or quick reuse of components. Sure, SDI and AES/EBU audio links are routable, but this does not constitute a loosely coupled system.
Loosely coupled designs rely on the aggregation of distributed services. A service is a networked application function available for use by others. Examples are video transcoders, media encoders, transfer engines, 3-D render engines, a media asset management (MAM) searching engine and technical QA evaluation. Each function is available over a networked IP connection, is instantiated on an application server, provides a standardized access API and defines a performance QoS for each client user.
Given a collection of relevant services, a designer can configure them to perform useful operations. These services are scalable, reliable (based on methods explained above), reusable (the mantra of services design), and well-defined in terms of API interfacing, data structures and performance QoS.
One outstanding example of a services specification is defined by the W3C.org group. It has defined what is called Web services, and this includes specifications for all aspects of interfacing to services, securing, naming, addressing, finding, monitoring and so on. Despite the name Web services, these are not bound to the Web. They may be implemented across the Web or inside a secure enterprise environment. Google, Yahoo, Salesforce.com, Amazon, MySpace, and many others offer Web services (or similar RESTful services) for public and private use. Some enterprise media facilities use Web services for select applications. They are not a panacea for all aspects of a media workflow. Real-time A/V operations often require dedicated devices to meet their QoS specification, and Web services fall short today. Still, when real time is not an issue, workflows using Web services are practical and will see more light of day as the industry builds confidence in their practical value.
Thinking SOA
The SOA is broadly defined as an architecture of loosely coupled, wrapped services communicating via published interfaces over a common middleware layer. This definition is not universal; there are others, but this will suit our needs and scope. SOA is strongly linked to Web services and other related networked service concepts. It provides the discipline to create scalable, reliable, managed business and media workflows. Using the technology principles of service reuse, granularity, modularity and interoperability, an SOA provides a firm infrastructure for efficient applications deployment.
SOA is not a technology, but rather an approach — a framework for creating solutions. Its acceptance is growing in medium- to large-scale enterprise organizations. According to the analysis firm Forrester Research, at least 63 percent will have one or more SOA implementations by the end of 2008. Looking forward, many media organizations will be using SOA principles. This affects equipment design because vendors will begin to provide service interfaces on their products to better interface into SOA environments. In the end, this creates more flexible workflows and efficient facilities.
Conclusion
The definition of workflow as “a set of sequential steps needed to complete a job” is deceptively trivial. Yet, behind these few words lies a world of sophistication in terms of design, planning, documentation, implementation and operational aspects. By using these concepts and maxims as a guide or checklist, architects, designers and engineers will have additional confidence in the merits their workflow solutions.
Al Kovalick is a strategist and fellow with Avid Technology and author of “Video Systems in an IT Environment — The Basics of Networked Media and File-Based Workflows (second edition 3/09)” (www.theAVITbook.com).
Editor's note: This article was previously published in the November/December, SMPTE Motion Imaging Journal, copyright, 2008, SMPTE.
