The move to file-based workflows has given content producers an unprecedented degree of freedom and flexibility while simultaneously offering them the opportunity for improved delivery times and cost savings. However, as is the case with the introduction of any technological change, file-based workflow implementation creates its own series of challenges, which in turn must be remedied by complementary technologies. In particular, challenges such as more digital assets, more distribution points and increased pressure to turn around productions in less time all contribute to the necessity for content producers to maintain a greater level of control over their content at any given time.
Moreover, archives are getting larger and disorganized, more versions of content are created for more distribution platforms, and metadata is becoming less consistent and more complex. In such an environment, manual resources are proving too imprecise and expensive to employ in order to address these issues.
Built on Adaptive Video Fingerprinting technology developed at Germany's Fraunhofer-Gesellschaft, the iPharro MediaSeeker Core Platform offers content producers and owners a noninvasive and highly customizable approach to content identification and, in turn, enables them to identify those assets at any point in the future. Created with a combination of core fingerprinting methods, an extensible use case framework and an easily accessible Web Service interface, the platform also operates on economical industry-standard PC hardware. It provides a foundation for maintaining control over growing volumes of digital content and enables effective monetization of that content.
How video fingerprinting works
A video fingerprint is a data file in which the audiovisual characteristics of a piece of video content are stored. Computer vision techniques are applied to material sampled at a given temporal parameter, with more intensive fingerprinting requiring more frames over a given time period and less intensive fingerprinting requiring fewer. The fingerprinting engine looks for a series of characteristics, such as color, contrast and borders, that are present in the sampled frames and reduces this information to its minimal representative form. This unique numerical representation of the fundamental characteristics of the set of sampled material is its fingerprint.
As fingerprints are created, they are used to populate the iPharro Database Indexing Server (DIS), which in turn serves as the reference for content identification. Queried media content is fingerprinted, and the extracted fingerprint is then compared against those stored on the DIS. Every distinct video has one unique fingerprint, so even slightly altered versions of the same video will be recognizable by their slightly different fingerprints.
Stored fingerprints typically are indexed and mapped into catalogs, much like directories, that help to speed the matching process. Either a piece of media or a fingerprint can be submitted for analysis and matching against video fingerprints within the DIS. The speed at which analysis is performed depends on the processing resources dedicated to the task, but in any workflow, the iPharro system delivers results many times faster than real time.
Benefits
Given the transparency and speed of video fingerprinting and analysis, this approach to content identification can reduce reliance on manual tasks and improve the efficiency of digital workflows. For example, if a content producer wants to know if a piece of footage appears in a content archive or on TV, the producer needn't look for it manually but instead can fingerprint it, store the fingerprint and automatically search for those characteristics in media from either of these sources.
As distribution venues for content continue to expand, the broadcast and media industry must create multiple versions of content for language dubbing, subtitling and content ratings. It is precisely in these areas where digital fingerprinting is of great assistance. It is often the case that a single program may have between 15 and 60 versions. By fingerprinting multiple pre-existing versions of various assets of differing quality, identifying their differences, and storing only the high-resolution master file and these “deltas,” media companies can reduce storage space and quickly and automatically create high-resolution versions that conform to the requirements of any distribution platform.
Fingerprinting technologies are not created equal. Most video fingerprinting providers offer services focused on downstream content identification designed to prevent content producers from losing control of their valuable assets once those assets have been released into the marketplace. Such technologies are able to identify content from a large content pool or single source with limited precision. What content producers need in order to assure that their content is organized, comparable and ultimately future-proofed is an adaptive fingerprinting method that can incorporate both precise frame-by-frame searches and rapid, large-scale searches. In other words, the fingerprint user should be able to customize the type of fingerprints being used to meet the demands of the particular application.
The platform architecture easily manages different types of fingerprints, and different matching algorithms can be customized to specific requirements or use cases. (See Figure 1.) For instance, advertisement monitoring requires indexing of a relatively small amount of reference material, on average about 20,000 clips of 20 seconds each. The volume of query material, however, typically comprises several channels recorded on a 24/7 basis — a potentially unlimited amount of data. In this use case, the platform can be configured to extract the query fingerprint only on the I-frame sampling level, enabling faster decoding, while the reference fingerprint is created using all frames. When an enormous amount of reference material, such as an archive, is being compared to query material made up of single small video clips, the model can be reversed, with reference fingerprints extracted from the I-frames and the query fingerprints from all frames.
Integrated easily into any pre-existing workflow, thanks to a Web Service API, the iPharro MediaSeeker Core Platform automatically fingerprints in-progress or finished (distribution-ready) content. (See Figure 1.) These fingerprints are stored in a central database/metadata archive, and they remain constant even as the media is sent downstream for transcoding and as metadata is reconformed for the requirements of the specific video sharing or platform provider. Ultimately, the platform could be employed to scan a library of properties, including media archives, Web properties or TV, and determine where a copy of the footage is present, be it the entire copy or merely a short subsequence.
Launch the process early
The implementation of file-based workflows and the digitization of tape-based archives offer broadcast news producers and other content producers the perfect opportunity to begin proactively organizing, streamlining and ultimately protecting their content. Launching the fingerprinting process early in the life of a media asset or the archive holding it is crucial because it precludes the later need for further content identification mechanisms. By making video fingerprinting a part of their digital media workflows, media companies not only increase the efficiency of content management, but also protect the continued value and integrity of that content.
Joshua Cohen is CEO and René Cavet is head of research and development at iPharro Media.
