LONDON—The image of television archives and attitudes toward the business of archiving itself has changed dramatically over the last 20 years. When videotapes were digitized, there was still the thought that the programs were going into a deep archive.
Today, the explosion in channels and the need for legacy programming to fill schedules has brought about a change in how archive material is managed. The ubiquity of the cloud is also making an impact, although not to the extent that might be expected.
‘ACTIVE ARCHIVES’
Deep archives—the original conception of an information store where material could sit on a “shelf” indefinitely without being touched—still exist but the active archive has become a viable method of storing programs and other data. Active archives work on the same basic principles as their long-term counterparts but can be accessed more frequently. They work as an online resource and can be based in conjunction with any main storage options: flash disk, tape or cloud.
This concept is advocated by the Active Archive Alliance, organized as a “vendor-neutral” information resource to guide people in the design and implementation of modern archives. Celebrating its 10th anniversary in 2020, members already included Western Digital, Spectra, Fujifilm and Strong Box but last year saw a flurry of new additions to the ranks, including IBM, Quantum, Object Matrix and, just before Christmas, XenData.
Molly Presley, head of global product, brand and customer marketing at Qumulo, is a founding member of the AAA, and told the InsideHPC blog last February that the reason for founding the alliance was the “recognition of the rapidly increasing value held within unstructured data.”
AGILITY AND SPEED
The means of holding all this valuable material—be it LTO tape, disk arrays or the cloud—is well established. The element that enabled active archives—and which will drive archiving into the future—is the ability to find and move the data quickly, safely and efficiently.
A leader in this is EcoDigital (formerly Front Porch Digital), which was acquired by Telestream in October 2020. Geoff Tognetti, senior vice president of Telestream’s content storage management business unit, says that the DIVA software suite, “complements” Telestream’s “broader workflow tools” for distribution and orchestration.
Tognetti explains that EcoDigital’s systems are used to turn rich media into independent objects, which can then be moved around as needed. He observes that there is “still a strong need for data tape,” which forms a part of the active archive. “It’s not just a place to store stuff,” Tognetti says. “It is constantly used as a pool of material, with data being put on and taken off tape every day.”
In parallel to this, content objects are also sent to the cloud but Tognetti feels there are still aspects of this decentralized storage option that need addressing, specifically ingress costs.
“People are not charged to put data in but they are charged to get it out and move it around,” he said.
There is also the issue of data gravity. “Broadcasters have typically got 10 to 100 PB of storage, which is a big center of gravity,” Tognetti added. “This can be on tape or as objects in a near-line store on premises.”
THE DIFFERENCE BETWEEN CLOUD AND ON-PREM
For Savva Mueller, vice president of product management at Masstech Innovations, LTO tends to be “most popular” on premises and sees the current big trend in storage for media and entertainment as an “accelerated move” towards the cloud.
“We have had some companies expressing interest in cloud archiving,” he says. “Pre-pandemic, most were still hesitant and very much in ‘wait and see mode,’ but now there is a greater acceleration and customers are seeing the cloud as something for 2021, 2022.”
Despite an increased interest in cloud archiving, Savva says potential users need to be reassured that any of their established workflows will be able to operate with the “new” technology.
“Cloud storage is fundamentally different to on-premises,” he explains, “so users want to be reassured that any processes they already have in place will be able to work in the same way as much as possible.”
The main public cloud providers—AWS Elemental (owned by Amazon Web Services), Google Cloud Platform and Microsoft Azure—operate in similar ways, with tiered storage so that retrieval—which can take from between a few hours to one or two days—does not have an adverse effect on customers. A criticism levelled at the big cloud companies is that there is little or no opportunity to move data between different platforms or select specific tools from one supplier to use on another’s cloud. Mueller, for one, sees no reason why customers should not be able to use the feature of a cloud vendor even if they do not use its deep storage.
Signiant CTO Ian Hamilton says that although “there is no doubt that everything is moving to the cloud,” customers in the interim are turning to more “hybrid” operations.
“Anyone with more than 100 PB of storage should be considering the cloud,” he said. “And if you’re already in the cloud, why move data back and forth? There are the costs of moving material around but also benefits, such as not worrying about the choice between tape and spinning disk.”
Hamilton observes that the private cloud is a much more media-specific option but in general, archiving involves a lot of duplication to ensure adequate backup.
“There is a lot of redundancy in storing information and not every storage system is optimized for that,” he said. “But the big vendors can optimize at that level. We’ve seen the private cloud come into and go out of fashion over the past few years.”
THE IMPORTANCE OF AI
The pandemic has had a wide-ranging impact on broadcasting, prompting storage customers to consolidate their archives, according to Eric Bassier, senior director of product marketing at Quantum.
“With less content being created, because of Covid restrictions, companies are looking to figure out how to repurpose and remonetize content from the archive,” he said.
To enable this to happen efficiently, Bassier envisions new technologies and business models coming together.
“AI techniques and services are giving customers different ways of enriching content in their archive,” he says. “AI can also make archives more searchable. The algorithms are pretty good at recognizing the face of a celebrity in a single frame of video, which saves time having to manually search the archive.”
Quantum is also expanding its service offering, in response to what it sees as some of the downsides to the public cloud.
“It is expensive to run and customers give up control of their content,” Bassier says. “So we are building a private cloud and selling it as a low-cost service. This is something of a change for Quantum but we have to transform how we go to market and sell to our customers.”
Which shows that, somewhat ironically, while the purpose of an archive is to look back, the technology and business of archiving is certainly looking forward.
