Orange of France recently became the first major European operator to tap into the fast growing phenomenon of companion screen interaction with content showing on the primary TV set. Orange launched an app for the Apple iPhone enabling TV viewers to check in to shows they are watching, and earn points or badges if they do it regularly. The app also allows users to access Facebook's social graph to connect with friends and recommend programmes.
This may not sound wildly particularly exciting or groundbreaking, following in the footsteps of existing app vendors such as GetGlue, IntoNow and Miso, which all offer similar check-in based social features for TV viewers. But it is only scratching the surface of what is possible with companion screens and opens the door to new apps that exploit closer synchronization. In fact Civolution, a vendor of video watermarking technology from the Netherlands, has become a cheerleader for the companion screen, highlighting its potential for revenue generation both through advertising and stimulation of online activities such as betting and click to purchase associated with the primary content.
Civolution initially positioned its watermarking technology primarily for content security applications through codes embedded either into the video or audio, but then realised it had potential enabling companion devices to tune into the primary TV. While video recognition technology such as Orange is using allows recognition of a programme as a whole, it does not facilitate identification of specific scenes or keywords. Video watermarking using Civolution technology could be used to insert such keywords into the sequence but would require the device to be pointed at the TV. Therefore Civolution recommends audio watermarking since this allows specific inaudible codes to be inserted into the audio stream and recognized by the companion device anywhere in the room without having to be pointed at the screen.
An alternative approach is audio fingerprinting, which enables the companion devices to recognise spoken words or sounds already within the content, but this is at the mercy of the soundtrack and can suffer from background noise interference. Furthermore fingerprinting does not necessarily yield appropriate cues for action. With watermarking specific codes can be inserted into the audio to trigger actions on the companion devices, which in the case of early applications will invariably involve access to the Internet. This could be for click to purchase an item shown on the main screen, or an invitation to place an online bet relating to a live sports event being broadcast at the time.
This does raise some questions, for although use of companion screens has the advantage of not interrupting the action on the main screen it can still be distracting. However there is the potential with watermarking to trigger delayed actions, such as inviting purchase of an item after the programme is over.
There is also cost involved at the production stage inserting appropriate watermarks within the audio. It is not clear at this stage how easy it will be in practice to edit the audio to insert watermarks subsequently for a specific companion screen application.
For this reason there will also be an important role for audio fingerprinting despite its drawbacks. It at least will work with the audio as it is, and can be used for example by a consumer electronics company to deliver applications exploiting companion devices without requiring cooperation with a broadcaster or content owner.
Meanwhile producers of live content, including sports and games shows in particular, are already considering how to exploit watermarking to increase engagement through companion screens. For example the companion could be used to provide ongoing feedback of how a TV poll relating to how audience polling related to a live show is going, which may persuade viewers to change their mind over their vote.
According to Civolution CEO Alex Terpstra some content producers are already starting to incorporate companion screens directly in their programming. It could then be that future games shows even require a companion screen to participate fully.
