Cognitive Mill™ — Automated Video Cropping Tool to Boost Your Social Media

It is as simple as it gets: you want to gain recognition and expand your reach to the audience — you go to social media.

And with social media nowadays being all about the mobile experience, it is crucial to provide quality content adapted to fit mobile screens.

In this blog post, we will talk about how you can automatically crop huge volumes of video to share them on Instagram, Snapchat, YouTube Shorts, Facebook, and TikTok — and how video cropping tools can help you do that in the most effective manner.

Traditional Video Cropping Doesn’t Cut It

As of now, the video viewership on mobile is at its highest: there are over 1 billion hours of YouTube videos watched daily, 70% of which come from mobile devices.

Twitter also claims that video content attracts 10 times more user engagement than regular tweets. Now, the watch-time on the platform increases by 84% yearly.

Therefore, the urge from many large broadcasting companies and streaming platforms like the NBA or Netflix to get social media traction with their video content is obvious.

But that opportunity also creates challenges.

The first one is effectiveness at creating content for social media. It takes much more effort to adapt loads of content for different social media platforms and regularly post on there.

Is there anything we can do with that workload?

Well, a large chunk of it can be cut with the help of automation: a video cropping tool that automatically crops a lot of videos from landscape to portrait aspect ratio as quickly as possible and then shares it on several social platforms.

Here, another challenge arises: to capture the essence of landscape footage with a cropped vertical video, the video cropping tool has to accurately identify the most important part of each scene.

Depending on the situation, the footage might have different things happening in different parts of the scene. Static image cropping may lead to a ton of quality content being left out of the focus area.

Here’s what we can do to solve those problems.

Tackling the Problem With Cognitive Mill™

We can rely on AI to do the heavy-lifting for us. There are video cropping tools out there that can automatically crop video using AI-driven technology to figure out the best focus placement.

Our automated video cropping tool, Cognitive Mill™, takes adapting video content for social media to the next level.

The tool relies on cognitive computing technology to imitate human focus when identifying the video’s salient content. The video crop software leverages huge volumes of data from a variety of sources to provide the most accurate result.

Let’s take a look at what sets Cognitive Mill™ apart from the competition.

Cognitive Mill™ Features

Cognitive computing algorithms

Cognitive Mill™ imitates the way in which the human eye focuses on particular areas of the screen during video playback, so the content you get catches the essence of the original video with the same accuracy as the human would do. The difference is that our automated video cropping tool can process large volumes of video content much faster while retaining that accuracy.

Adaptive focus

As the salient content moves around the scene, the cropped frame changes its position to capture everything important. The frame moves smoothly along the scene, making for a seamless viewing experience.

Content awareness

Cognitive Mill™ can also work with different types of video content with no additional tuning. Whether it is a TV show, a movie, a dynamic football game, or a news update — the video cropping tool will easily recognize and crop the most salient parts.

Integration through the API

You do not have to jump back and forth between your video production workflow and third-party video crop software.

We seamlessly integrate our solution with your video production tools through the API. This way, you can build the tool into your pipeline as another step to your video processing workflow.

So, you capture the video stream of a sporting event, get your highlights cut and all. With Cognitive Mill™, you can analyze the video and crop the aspect ratio to fit the mobile screen automatically. No more tedious work and countless hours in front of the screen.

But this is not all.

We also provide social media integration using respective APIs, so you can push your content to the platforms you need in a matter of seconds.

All of those features make cropping landscape video to portrait aspect ratio a breeze. The whole process is automated and therefore boosts the effectiveness of your video production.

Now, you can easily engage your audience through social media and expand your reach with almost no effort.

Video Cropping Competition

There are a number of web applications that allow users to automatically crop videos for social media like Instagram, Snapchat, TikTok, and Facebook.

Aside from that, the web apps act as basic video editing software that can be used to add captions, adjust the aspect ratio to fit different smartphone screens, and preview the footage.

But let’s get back to the flagship feature of those services — the cropping part. The apps claim to crop the footage from landscape to portrait format without any help from a human.

The question is: how do they know what parts of the scene should be in focus, and what should be left out?

For the most part, they rely on AI-powered object tracking to find the most important part of the footage and crop it accordingly.

Let’s study the way they work.

Google’s video cropping framework

We can do so by using the open-source Google’s AutoFlip framework as an example:

When analyzing the footage, the technology detects composition changes, or cuts, to identify different scenes;

After that, it goes through each scene individually and applies object tracking to find salient content;

Then the tool crops the scene to include the important content in the frame.

In case the object tracking fails at recognizing the right object to focus on, many online video cropping tools allow for manually adjusting it.

Basically, the objects are recognized with the help of run-of-the-mill Artificial Intelligence, and the change of context is handled by breaking the footage down into scenes and individual analysis of said scenes.

And that is the problem.

In many situations, simple object tracking may not be enough to recognize the most important parts of the video. For instance, if the scene has several objects moving at the same time — the essential portion of the scene might get cropped out.

As you can see in this video, the object tracking that Google utilizes with their AutoFlip framework fails to follow the salient content if several objects are moving. On the other hand, Cognitive Mill™ correctly recognizes the objects worth focusing on.

Cognitive Mill™ applies human-like contextual understanding to draw decisions on what content to crop. It can also work with even the incomplete set of information and still provide an accurate result.

Here’s how it happens.

By leveraging mathematical motion analysis paired with Cognitive Perception, the tool is able to thoroughly analyze the motion profile of objects in relation to each other, the location of objects in the scene, and the scene context. By doing so, it effectively pinpoints the most salient content and crops it.

Whether it is an action movie, a dynamic football game, or a comedy special — Cognitive Mill™ accurately recognizes the objects worth focusing on and keeps the frame on the relevant content throughout the playback.

Providing the Most Value for Media Companies

Since the major part of social media traffic is composed of smartphone users, the media companies are looking for ways to automatically adapt their video content from landscape to portrait aspect ratio and post those to all social media in one shot.

But most of the automated video cropping tools do so with the help of regular object tracking that may struggle with recognizing the right focus areas in complex scenes. Therefore, the essential content might be cropped out.

To combat that, we apply human-like contextual understanding and thorough motion analysis to identify salient parts of the scene and crop it accordingly. Powered by Cognitive Perception algorithms, the accurate result is delivered within minutes — for instance, Cognitive Mill™ chews through an hour of footage and crops it in just 5 minutes.

If you want to streamline your video production process — drop us a line at support@cognitivemill.com or fill in the contact form below. You will get your system running in a few days with no additional setup!