Four researchers at the University of Washington have collaborated with Google and Facebook to turn video into fully interactive holograms.
The team of researchers have developed a machine learning algorithm capable of transforming 2D football clips into 3D reconstructions.
They use video of matches, track the movements of each player individually, and then map the players into 3D models. These 3D models have been extracted from the video and projected as a 3D representation onto a football pitch on a table top - allowing the audience to watch the match without the need of a screen.
Viewers can watch the game from multiple angles and the 3D replay will also be available a few minutes after the match ends.
“Our goal is to enhance the viewing experience of sports,” Konstantinos Rematas, one of the researchers on the project, told Digital Trends. “Instead of watching a soccer game or highlights on a flat 2D screen, we convert the original video into 3D and visualise it in augmented reality. Essentially the game becomes a hologram, where you can move around and look from different viewpoints, generating a more immersive experience.”
“The next steps are about increasing the quality of the game reconstruction,” Rematas continued. “In particular, we want to estimate precisely the location of the ball and reconstruct better the players — occlusions, full 3D shape estimation, [and more].”
You can read the researchers' paper here.
