Maybe this has happened to you: You’re on deadline for a live, episodic show, and need a few seconds of archival footage. Except that it’s somewhere else. And even though you put in the request a few hours ago, there’s a wreck on the road between it, and you. Plus, it’s on tape, which can make ingest unpredictably dicey -- which always seems to hijack your day right when you’re on a super-tight deadline.
The tape finally arrives, but searching for the sliver of footage you need is painfully linear: First, the dang thing gets jammed in the machine, during which you learn that there’s not even a spreadsheet or simple manifest database to guide you.
And then you find out that the guy who knows all the double-dog secret workarounds of the nine or so vendor-proprietary subsystems, necessary to get your segment going, is out and unreachable. (And it’s only 10 in the morning.)
So You Have Archive Issues?
The common denominator in all of these day-to-day broadcast realities can be summed up in two words: Archive issues. Archive issues that hobble the ability to emulate the agility of Internet-sourced video. Archive issues that exacerbate storage capacity, aggravate operating costs, and lag viewer expectations.
Archive issues are an active undercurrent right now, in broadcast industry conversations -- in hallways, over dinner, and in cross-departmental meetings between “ops-side people” and “IT-side people,” often within the same organization. It’s a daunting and communal challenge, all centered on the need for “active archives,” or at least some way to scale-out those archives, once and for all. On their cloud, or a public cloud, or some combination of the two.
Cisco UCS S-Series servers deliver a scalable, modular active data storage solution that's ideal for media. They offer storage that’s less expensive than a NAS, but 100x faster than digital tape alternatives used for archiving. Check out the UCS S-Series on the Cisco website.
Here’s the short list of “how best to” considerations, culled from my notes with multiple conversations with broadcast-side engineering and IT colleagues over the past few months. “It’s the same for all of us,” one explained. “It’s all these questions about ‘how best to’ do this or that. In no particular order:
- How best to “scale out storage,” as an over-riding driver.
- How best to solve -- or at least stay reasonably ahead of -- video resolution advances, like the progression from HD to 4K to 8K (with 16K in the wings), each progressively “heavier” to carry.
- How best to solve for the growing landscape of display screens, beyond the TV.
- How best to harness the celebrated “ease of use” inherent to “the Internet” -- on the back end, and the viewer-facing front.
- How best to make something truly “evergreen” -- for real, and forever. This is closely related to the matter of how to forever avoid the need for yet another “data migration,” from one tape or digital method to the next.
All of this is happening within the larger, organizational conundrum of how to slipstream into that new cultural norm that is agility, DevOps, and “mobile speed” -- itself a gear or two faster than “Internet speed,” which is several gears faster than traditional, broadcast speed.
Here’s my short list of the “perfect world” for archive management, from the engineers, technologists, and data/IP-side people who work in media and entertainment. The specific ask was, “what characterizes your perfect world, when it comes to your audio/video archives?
Active media archive solutions offer broadcasters options that are significantly less expensive and more flexible than traditional methods.
The Perfect World
Desire #1: Remove the shackles of archival storage capacity! On tape and in file-based environments, said one senior broadcast engineer, adding, “every time you add another tape to the library, you’re adding another mouth to feed.” A few years later, that tape will likely need to be handled again, she noted -- or worse, become obsolete.
The disk equivalent of this also exists, by way of “NAS proliferation”: You fill up one system, then another, and another, until you feel like you can’t manage anything, because you’re so busy managing everything. The point: Tape is inexpensive, and limitless! File-based storage is not as inexpensive, but it’s also limitless! In both cases, the more there is, the larger the management headache. Short-term fixes often cause long-term nightmares.
Desire #2: Make it as easy to search an in-house archive for a dated segment, as it is to search on Google for the nearest fish taco. Shifting away from tape, to a private (meaning in-house) or public (meaning AWS, Azure, Google, pick your favorite) cloud simultaneously brings web-like advancements in segment searches. Systems already exist that offer “self-describing media,” for describing clips, dates, anything that exists on the “box art.” Make it as easy as a web search, another broadcast network engineer said, “as opposed to today’s methods, which are about one Dewey away from having to go to the library to flick through a card catalog.” Simplifying search can save money, too, noted one media archive manager, who confessed to sometimes buying the same stock media more than once, “because it’s easier than finding the original.”
Desire #3: Give it some AI, for heaven’s sake.Artificial Intelligence systems thrive on the mountains of data that look daunting, otherwise, and the archives of the world’s broadcasters are … mountainous. (Metaphorically, anyway.) Consider a hypothetical but plausible broadcaster with a show that’s entertained millions of people, every week, for 40 years. Let’s say that’s about 800 episodes, archived on thousands of tapes (analog and digital), and on network-attached drives. All in, the archive is measured in Petabytes. That’s a lotta headache.
The good new is, work is also progressing, very rapidly, to apply AI to automated metadata creation. Its role: To gather and sort known (embedded/preexisting) and unknown data, on a frame-by-frame basis: That’s a dog. That’s Tiny Fey. That’s a red Jeep. Such are the hooks available with active archives and AI.
Desire #4:Make it slipstream right into my existing workflows. All of them. Over and over again, a common refrain when daydreaming about the perfect archival situation involves the day-to-day, very busy, very hectic workloads shouldered by today’s broadcast and IT professionals. Nobody needs or wants “big, monolithic system #9,” which requires a squadron of integrators to get it tied into existing back-office subsystems. An ideal active archive system spans objects and files; media asset managers and editing stations. All of it, wherever it is, with a front-end that speaks the same language as today’s clouds.
Desire #5: Make it break the “chains of dependency.” Particularly with technologists working with object-based storage, a strong desire persists for standardized ways of handling archives, “with HTTP-based everything.” Make it work with the storage protocols common to the Internet, like the S3 API, among others -- which were created for the Internet. Not adapted, shimmed, or otherwise derived from existing systems. Nothing proprietary, with objects that are portable and can live in any storage environment. (There was considerable “harrumphing” on this one.)
These are the challenges facing lots and lots of engineers, technologists, and IT-side professionals in the media and entertainment sector. They’re at the “how best to” phase now, and sizing up available ways to shore up their mountains of archives, once and for all. Their wish lists are understandable, and doable. (So I keep hearing, anyway.)
Watch for “active archives” to be a common thread over the next couple of months, as broadcast-side professionals start gearing up for the 2018 National Association of Broadcasters. And, if this kind of conversation is happening where you work, know that you can get a much more detailed assessment with two companies who will be there: Cisco, and its partner Cloudian. This is as close as I’ll ever get to an endorsement, by the way, but from the Department of Full Disclosure, they commissioned this series, and provided the background and some of the engineers who informed this piece.
In closing: May your archives be active and scalable!
This is the second in a six-part blog series preceding the 2018 NAB Show. Click here to read more about what broadcasters will be talking about at this year's show. Click the thumbnail below for the next installment in this series.
About the Author: Leslie Ellis is a respected “technology translator,” well known in cable and telecom circles for her award-winning, 20+ year “Translation Please” column in Multichannel News. She took on this Cisco-sponsored pre-NAB series to point out common and frustrating obstacles, for anyone on the sliding transition toward “being more Internet-like.” It is less of a comprehensive representation of available options, and more a glimpse into what’s worrisome, on a day-to-day basis, for engineers and IT people who work in media and entertainment.
