Disasters and recovery plans are not new concepts to the broadcasting industry; its history is full of events that have temporarily shut facilities down. Throughout these events, most broadcasters have managed to function, providing news coverage and entertainment programming. But disaster planning and business continuity are critical. The longer it takes for normal business operations to resume, the less likely the business will survive. Many of the businesses that do not have business-continuity plans do not survive a disaster. The challenge is to plan for every possible contingency.
Disasters take many forms and come from many different places, but they all have a common effect: They impair your resources. The trick is to minimize the impact, and then restore operations back to normal. You can do this by developing alternates and work-arounds, testing them to make sure they work, documenting them so that they will work in your absence, and keeping them up to date with changes in technology, your facility and its needs.
Lightning from a powerful thunderstorm strikes part of an electrical power grid. Such strikes release billions of watts of electricity, which can easily overload portions of the grid and cause widespread power outages.
Starting to plan
To initiate a disaster and business-recovery plan, start off with an honest risk assessment. To what risks are you exposed, and how likely are they to happen? Many stations have experienced disasters and have plans and equipment in place to deal with them. (See the sidebar for some examples of the types of risks you might encounter.) Many stations have some backup equipment such as second transmitters and antennas, but have not reviewed the need for further backup equipment. There is a school of thought in some broadcast business offices that it is cheaper to insure against a loss than to try to prevent it, so some broadcasters purchase insurance rather than extra equipment to get back in business. This increases profits temporarily, and makes owners happy. But insurance costs are rising, and some coverage is unobtainable at any cost.
Some departments, such as engineering, might have a false sense of security, and see themselves as immune to some risks. For example, let's assume that everyone at a station agrees that the responsibility of the traffic system belongs to the sales department. But, if the traffic system goes down, master control will soon run out of programming, which will affect engineering. And any department can experience the untimely departure of a key person, which means that others in the department will have to carry extra responsibilities until that key person is replaced.
Another issue that crops up in times of crisis is that a clear delineation of responsibilities is sometimes lacking, which can separate resources from people who need them. Some resources, such as office supplies, can be handled by one person, with someone else acting as a backup. But trying to locate a suitable studio may require a team approach, with each member bringing in an area of expertise. For example, engineering can be responsible for focusing on issues such as access to satellite receiving while the business office focuses on the legal issues. The important point here is that you must decide these issues before you need the resources.
A tornado rips through a community like a chainsaw through a miniature village. A tornado’s formation and path, unlike that of a thunderstorm, is very difficult to predict.
Your viable disaster-recovery plan must include a list of the resources — utilities, equipment and supplies — your facility uses, along with documentation on how you normally obtain them, how else you can obtain them and, when they are temporarily unavailable, how you can make do until you can get them again. Start with the obvious things, like electricity, skilled labor, and technical and office equipment, and then consider more unusual items, like the building itself. As you make a list of each item, document how often it breaks or depletes, how you could replace it, how long it would take to replace it, where to get the replacement and how you would transport it, what else is affected by its loss, and what might place an unusual demand on this resource. Most disasters don't create a total loss, so having alternate resources can help you make a substantial recovery. Some items will stand out as being important to your operation — perhaps because they take a long time to replace, or because their absence makes processes more difficult. The question for those items then becomes: How can you reduce the chance of losing that resource, or reduce your dependency on it? If your phone service fails often and you lose the transmitter link, can you get the programming to the transmitter some other way? Perhaps documenting losses caused by old equipment will help get replacements.
Backup
Uninterruptible power supplies are a station’s first line of defense against disasters, both natural and man-made, that cause power outages.
The amount and kind of backup a facility should have varies from one facility to the next, and seems to contradict the current wave of consolidation. The real question is: How long can your station afford to be out of business, and how likely is it to happen? You need to weigh this against the cost of maintaining a second facility, and the possibility that it will suffer problems at the same time. From that, you can subtract the amount of daily work you can accomplish there. For example, if you put your small studio to productive use by producing a show that is more convenient for the people involved, then you can have one fewer studio in your main location and have a backup facility for your radio station at a minimal cost. It will need to have some extra equipment, but a lot of the major costs are being used for an ongoing part of your station's daily operation. Maybe keeping one of the old station's facilities around makes sense in today's world.
Much of the technical equipment used in broadcasting, such as RF components and production equipment, is unique to the industry. This equipment tends to be expensive, and generally there are no spares standing by. While you probably will have an auxiliary transmitter, perhaps even at a different location, the same cannot be said for a production switcher. If you have a second switcher, it is probably located in the same facility and is used at the same time as the main one. Is there a way that corporate can have some of this equipment available for all the stations?
In times of disaster, some people adopt an attitude that was prevalent in the wild, wild west: every man for himself. Suppliers that you had counted on to have things you consume may not have them. Repair people that previously would have come over on short notice may not be available for days. On top of that, your equipment may break more than usual due to the heavy workload on it. If you are in a disaster situation, your suppliers (and the suppliers' suppliers) may in a similar situation. They would have a higher than usual demand for their goods and services, while dealing with the same problems you are.
A snapped utility pole dangles helplessly after a storm. Don’t let your station suffer the same fate.
They also supply places like hospitals, local government and relief agencies and would give priority to those customers, leaving you in third place. While you can try to get promises of priority service, the reality is that suppliers will want to make sure that the emergency services taking care of the disaster have the materials they need, even if it means you don't get the supplies you need. This means that, in your recovery plan, you need an alternate supply source. A good place to start might be a co-owned facility in a nearby city. It may take longer than usual to have the supplies delivered from 100 miles away, but you might get the supplies sooner than waiting for your usual supplier. Then, if they are ever in a similar situation, they might be able to use your suppliers.
Availability
There are several things that can help you speed the recovery process. First, make the recovery plan and the tools to implement it readily available. In the heat of the moment, will you be comfortable scanning through your address book on your laptop or PDA, or flipping through pages in a notebook? Since part of the recovery plan is accounting for your own lack of availability, will all the other possible users of the plan be able to implement it?
A snapped utility pole dangles helplessly after a storm. Don’t let your station suffer the same fate.
How will it be kept current, and where will it be kept? Some of the information, such as employees' home addresses and telephone numbers, may be sensitive. Yet this information will be invaluable in times of crisis. Some departments may need to have their business continuity separated from the main station's plans and staging, yet will need guidance on the complete plan. The legal department may not want it widely known exactly where the backup copies are kept, but it will need to know where to send them in a crisis.
Test and update
Once you have looked at all your options, the next thing to do is to test them. For example, find out if the rental house really has the two cameras right now. Make sure the backup microwave really works. You need to test everything. And you need to familiarize the people who would be implementing the changes with the proper implementation procedures. Make sure that the new maintenance engineer can start the generator manually, that the master control operator can change transmitters, and that the tape operator can move and tune the satellite dish by hand. While you are testing your plan and training new personnel, ask yourself what else could go wrong. The exact training schedule will vary by station. The recovery plan and the people involved with it change, and operators forget. The plan is a living thing that needs to change as circumstances change. Having old information may slow recovery and increase the cost of returning to normal.
We hope this review has given you an idea of the size and scope of disaster recovery. The most important things to remember are that making information available and rehearsing the basics can keep your station in business. More than 50 percent of the businesses that shut down for 30 days never reopen. Don't let yours be one of them.
William Kirkpatrick is engineering manager at WABC-TV in New York. Views expressed in this article are not necessarily those of ABC.
Disaster characteristics
Disasters come in many forms and have many characteristics. It would be wise to take these characteristics into account in your plan so you can prepare for them and recover from them.
Local impact vs. regional impact. How large an area does the disaster affect? A fire next door to the station is a local event, while a blizzard is more regional. This will influence issues such as the availability of local resources.
Newsworthy vs. not newsworthy. An event such as a water-main break in front of the station may not be newsworthy, while a neighbor's evacuation may be. This will influence the news department's expectations and use of the available resources.
Predictable vs. unpredictable. How much warning, if any, will you have prior to an event occurring? Some natural phenomena, like tornados and hurricanes, are seasonal and, therefore, predictable to a certain extent. In North America, the summer months inevitably bring tornados to the midwest, and late summer and early autumn bring hurricanes to the Gulf, southeast and mid-Atlantic states. Specific hurricane events can be predicted days in advance. But other specific natural events like tornados and earthquakes are much harder to predict. Man-made events like accidents, crime and terrorism are largely unpredictable. The bottom line is that, the more predictable an event, the more time you have to prepare for it. If an event is unpredictable, you will have to be prepared for it at all times.
Preventable vs. unpreventable. What steps can you take to prevent an event? And what steps can you take to minimize the effects of inevitable ones? Man-made events like accidents, crime and terrorism are preventable to a certain extent. Natural disasters are not preventable.
Minimal impact vs. severe impact. How severely might a disaster impact your operations? Having limited resources will increase the severity of an event's impact on your operations.
Each of the above sets of characteristics has two conditions: one is bad; the other is worse. The key to returning business to normal is to minimize the duration of the occurrence and the severity of the worse conditions, speed your way through the bad portion and return to normal.
