Managing storage in a network environment is an ongoing requirement whether for broadcast facilities, data centers or even the home PC. Even the modest television operations center has to deal with thousands of inventory elements encompassing hundreds of hours of content. As a broadcast facility grows, storage requirements will span many different sets of platforms and storage subsystems. Administering these systems can become difficult even if an asset management system is employed. And if you're trying to manage it manually, then you probably need a high degree of support labor and time just to stay ahead of the curve!
MANAGING STORAGE
In the not too distant past, managing the storage on a video server platform involved two basic steps—purging aged content to make space for the new material and ingesting new content to fulfill programming for the immediate future. The latter also involved the replenishment of expired promo content with new material that retains the same clip identification or "house" number. Asset management was often a combination of manual actions initiated by traffic, the business side of the broadcast operations.
The storage system of today has become an essential business asset. As content delivery to local cache servers via satellite began, migration was managed by manual processes. Content would be played out as baseband video or as an FTP (file transfer protocol) to the broadcast air server. Today, content is moved via automation software instructions exchanged between the catch and broadcast server. This transfer had to occur before the catch server was purged by the service provider, which happened whether or not the transfer occurred and usually without local intervention.
Broadcast content could also be ingested automatically from linear satellite feeds under the direction of a record scheduler, which obtained its instructions from an application resident in the automation system with data generated by the traffic system. The scheduler would handle the start of record functions and the asset management tasks often with manual assistance from the feeds room operators.
In nearly all these operational cases, the procedure for recording or purging content on the storage system is a process that involved direct instructions from traffic. A purge list would be generated for the automation system and originating from traffic, and/or a manual intervention by a master control room operator that physically authorized the deletion of the content.
Fast forward a little and you have a real challenge addressing the ever-growing volume of content being stored on disparate platforms such as graphics systems, still and clip stores, broadcast servers and caches, browse/proxy servers, editing systems, etc. This is precisely the reason why asset management systems—call them DAM (digital asset management) or MAM (media asset management)—are surfacing in the enterprise. But this installment isn't going to address managing the physical assets, i.e., the "files" and their supporting metadata. This time we're going to look at the components and elements of managing the storage network itself.
OPERATION CRITICAL
The storage system of today has become an essential business asset. It doesn't much matter where the data is located or how the storage is configured, the management of that storage becomes critical to the overall operation. There are two fundamental approaches to storage management in a network environment. One is to manage storage through a server-provided mechanism. Another is the more common approach taken by storage industry companies, that is, to manage it directly through a storage product interface.
Given the emphasis on Internet computing, today's focus on storage management dwells on how rapid changes in configurations occur and how to support these continual changes throughout the enterprise. This mandates that storage management be flexible. It must have the capability to adapt to growth and continue to maintain access regardless of size—thus storage management system must be scalable.
Data must also be accessible and available at all times. For example, when the promotion department needs to marry graphics from a Mac, video clips of today's top stories from a newsroom NLE, and a canned bumper stored on the broadcast air server, they need to have the content accessible to their work platform and available when they need it. The production tasks should not be limited to drawbacks in the network storage system, even if their work location is outside of the main broadcast facility's central equipment core.
Finally, the storage network must be reliable. While transmission failures in a network are not uncommon and may occur for myriad reasons, and not necessarily desirable—in most cases a TCP/IP transfer that has to be restarted is not a major show stopper. However, if the storage network goes down or the database is corrupted, the ability to get any data from the system becomes catastrophic—impacting or even paralyzing huge portions of the enterprise. This is one of the prime reasons that broadcast operators hesitate implementing a completely centralized storage system founded on network practices, with the other reason being incompatibility of systems, files and the like (a topic for another time).
INFRASTRUCTURE AND LOGICAL COMPONENTS
These previously stated requirements (availability, accessibility, reliability, and scalability) are objective in nature. They can be impacted by dependencies such as cost, implementation and administration policies. However, there are certain areas that must be considered differently, ones that will allow for system growth and for the provisioning of the network services desired. These physical infrastructure and logical components are essential to supporting the overall network and should be apportioned properly.
By definition, the physical infrastructure is comprised of network cabling (or wiring), patching systems, connectors, host adaptors, and managed network switches or hubs. The logical components include applications for flow control and handling of network traffic, virtual networking, link aggregation and security. The physical cabling issues are rather straight forward, involving the proper deployment and testing of Cat 5e or Cat 6 UTP-components or fiber optics.
Don't cut short the capabilities of a network switch. In selecting a switch, understand the QoS (quality of service) feature sets. Advanced switches should have the ability to create VLANs and be capable of turning off certain undesirable protocols. Expandability and backbone (backplane) speeds are important for interconnecting the blades of a large switch. Spare components (i.e., blades and power supplies) as both active hot standby and shelf spares should be considered when a fully redundant configuration is not in the budget. Remember, these are mission critical, core components of your network and for complex systems must be thoroughly engineered for proper performance. The size of the files and the data rates incorporated require sophisticated systems that should not be compromised.
Another equally dynamic component of the storage networking concept is that of intelligent data organization. In the data-centric world the function is called "filing." By intelligent, we mean that the file system and database determine how data is stored and retrieved, and what metadata is included and associated with the data. Ultimately, how that information is represented to both the users and the applications are of prime importance, elevating the overall value of the information to the enterprise.
FILING FUNCTION
The management of the filing function is a subset of the overall network storage management solution. Filing, as a function, is a logical activity with no dependence on hardware. Some might argue that the file system, filing activities and the storage disks are all integral elements. That may be true, however, what we want to emphasize is the function of filing becomes independent of the types of disk drives employed.
Those remaining components in a storage networking architecture center upon the nonvolatile holding space for data that is reliable and stable. Here both logical and physical elements are employed. The more obvious being the physical magnetic or optical disk drives or in some cases the solid-state storage. The logical side includes the RAID structure, how mirroring is accomplished and the methodologies for making multiple drives appear as a single volume, referred to as "volume-management." All these elements must work integrally to form a manageable storage solution.
In conclusion, all the aforementioned elements, components and principal concepts must be properly considered and proportionately integrated in order to form an overall storage networking solution. The environment may be an open system, i.e., one that can store all forms of media on a common networked platform; or a closed system such as what is found in a NLE or video server designed primarily for housing moving media in mission critical applications.
It is entirely possible and often common practice to have both open and closed systems in an enterprise class facility as long as the limitations and expectations are understood and accounted for. How these systems interoperate and the tools necessary to manage the end-to-end activities remains the ongoing concern—one that may eventually overshadow video compression formats and studio operations entirely.
