Football and basketball highlights generation

Hardly anyone doubts that the sports industry is one of a significant scale and importance. We cannot imagine the modern world without sports broadcasts. Even throughout the pandemic, we are sure that there’s no way sport is leaving our lives. Yet, the industry is most likely going to be transformed forever by the disruptive 2020.

The latest sports industry trends breakdown by Deloitte states that the technological impact for fan engagement will be as high as ever. It is expected that there will be restrictions to in-person sporting events attendance. So the growing amount of attention should be given to the digital means of engagement.

Sports broadcasting seeks automation

There’s no need to mention that sports is a Multi-billion-dollar industry. Sports broadcasting is so too, with hundreds of teams involved in the whole cycle of content delivery to ensure the best quality of viewing experience. Everything has to be delivered in near-real time.

In the world where everything seeks automation, the sports broadcasting industry cannot be foregone. We at AIHunters are driven by the desire to automate media processing and in this way to reduce manual work in the Media and Entertainment industry.

The process of manual highlights creation is rather a complex one and is defined by the skill of an operator working behind a mixing panel. On the other end of the production cycle there are thousands of media, youtube channels, bloggers and all sorts of production studios who’s main area of attention is sports and highlights in particular. After a match is played, the race for the fastest highlight reel creation starts. And whoever manages to upload the compilation first, gets all the views.

It is an excruciating job — trying to outrun the time. That’s why we offer the opportunity to create highlights automatically with the help of our highly intelligent system. Automation makes it possible to get the highlights in a matter of seconds after a final whistle.

As the process of automated highlights creation is rather a complex one, let’s walk through it step by step.

Automated highlights creating system

Let’s take a look at how a scene is created. There are some stationary cameras on the field controlled by operators. Who, in their turn, track the movement of the ball manually. This attention area which is created by humans defines the input for our algorithm.

Our main goal technology-wise is to split a game’s timeline into semantically-defined parts and then to choose the best ones to compile them into a highlight reel. At the first stage, as said, we have to sort the scenes out. This means that after timeline segmentation we will have separate segments with, for example, only tribunes, only goal or hoop, only close-ups, only ads etc. This will all fall into the separate ‘clusters’ as we call them.

To achieve this, we leverage supervised and unsupervised deep learning. Then we finetune the system to understand the output. Now we know what cluster matches the particular scenes. For not broken semantic flow of the game, we apply self-similarity approach to avoid pure «picture based»segmentation, but to keep clusters much more coherent.

In our system, we achieved a well-defined precise clustering with such clusters as:

• Close-up (face)
• Full height
• Tribunes
• Whole field
• Goal
• Face and full body (including player face indentification)
• Face and field
• And many more

for football, and virtually the same for basketball.

The system also places into a separate cluster the following:

• Blocks of ads
• All types of repetitive graphics.

Besides, there’s the cluster for the moments when the system can’t determine what happens (unknown content that is unique to the game). This cluster is colored black and needs to be managed manually to ensure the preciseness of the information. The amount of time in this cluster is no higher than 7% of the whole game time and the key scenes are never included there.

The types of highlights

There can be different types of highlight reels applicable to the different business needs. For one, you want to feel the vibe of the game, with all the emotions, tribunes excitement, close-ups, coaches yelling and rending their hair. In other words, where the technical moments are not so important. In other, you just want to watch the core, most concentrated excerption of the main game moments.

To match both of those types of needs, our system generates two possible types of highlights:

Trailer like

This type is applicable when you don’t want to reveal all the key points of the match, but rather to convey the intensity of it.

To generate such type of highlights compilation we apply the concepts of information theory (such as entropy) to the already-split content parts to analyze the change of scenes and include only the best moments into highlights compilation. The AI-based ‘eyes’are looking at the dynamics of the game and the AI-based ‘brain’ decides on how to convey the rhythm, intensity and angles of the game to make a highlight the most exciting. And the frame-precise scene segmentation helps not to lose any thrilling moment of the game.

Our system guarantees to find the balance between the length of a compilation and its saturation, including the moments with the highest score of intensity.

More trailer examples at https://cognitivemill.com/solutions/sports-highlights/

Technical

This type of highlights includes all the goals and gameplay. This is a shortened version of a game created for a viewer to save a considerable amount of time by not watching the unessential.

To reduce the necessary viewing time even more, we apply a variational autoencoder (VAE) to football matches to cut out the moments of the least tension.

In the examples below, you can see how the viewing time of sporting events could be reduced.

More trailer examples at https://cognitivemill.com/solutions/sports-highlights/

Then we have a number of user-defined parameters such as the length of a compilation, the clusters you want to include, the amount of close-ups etc. that you can play around with according to your business needs.

To sum up, the system divides a game into clusters with the help of supervised and unsupervised deep learning, then leveraging the concepts of the information theory to create a perfect highlight. Besides that we use a VAE for football matches to reduce the necessary viewing time and user-defined parameters for both football and basketball games. This kind of system agility allows for a highly-customizable business-oriented highlight reel creation.

IN CONCLUSION

In the struggle to compile highlights reel before others, time really matters. The fastest post-game highlights producer gets all the views, leaving the others on the side of the road. Or to say it in ABBA’s well-renowned words, «the winner takes it all».

Taking the advantage of the AI-based automation makes it possible to create a highlight reel in a matter of seconds after a game is played. The customizability of the system’s parameters makes it simple to adjust the output to the needs of any business along the whole cycle of sports content delivery.

We encourage you to try it for yourself and see how magic happens AI automates even the most complex and creative parts of media processing