CognitiveReelz™ Sports
Product and solution overview
CognitiveReelz™ Sports is one of the Cognitive Mill products that offers you an outstanding solution for creating highlight reels for different sports events, TV shows, streamings, etc. It also analyzes the pattern of a video, summarizes video content, creates trailers, highlights, and more.
Our cognitive computing cloud solution watches and analyzes video content, then selects highlights as a human would, and does it much faster. It takes a few clicks to run the process and the machine does the rest. You can also select the types of highlights you want to include in the trailer before it is generated.
One of the core CognitiveReelz™ solutions is Automated Sports Highlight Generation. This solution saves you much time, effort, and money by automatically creating highlight reels of a sports event both live and recorded. The highlights of a live event will be ready in seconds after it is over.
Now the AI-powered cloud ‘robot’ is able to analyze and process the following events:
- Soccer matches
- American football matches
- Basketball matches
- Hockey matches
- Boxing fights.
The system analyzes video content, cuts the video into multi-level pieces, and then sorts and labels them according to the content type.
It recognizes and sorts separately game action and side content. In the game action part, the system can identify tense, high-drama moments, attacks, and etc., that can be used to generate the corresponding types of trailers.
You need to run a parent process to get output metadata first. Depending on the sports event, you should select the corresponding type of process: Soccer meta, American football meta, Basketball meta, Hockey meta, or Boxing meta.
After running a parent process, you can get a meta.json file that contains all the required metadata for further processing with MediaMill™ or with third-party software, or you can run a child Trailer compilation process and get a trailer.json file with metadata for different types of trailers sorted and labeled.
Benefits:
- You get metadata for creating highlight reels of a sports event in JSON format.
- You can generate a trailer.json file for further creation of a number of popular types of trailers.
- You can create highlights in the fully or semi-automated mode using the Cognitive Mill child processes or with the help of third-party systems and software.
- You can process both recorded videos and live streams.
- The video is processed in the fully automated mode without human involvement, which is faster and cheaper.
- You can interact with the platform either via the UI or via the API.
- We offer ready-made solutions for Soccer, American football, Hockey, Basketball, and Boxing. But our system is flexible and smart enough to adapt to other sports and you can request any other kind of sports according to your business needs.
Let’s review what is included in the output meta.json file after running the metadata analysis.
Output metadata
We’ve taken the analysis of a Soccer match as an example, but all other kinds of sports are analyzed the same way. And the output metadata you get is similar to the displayed in the JSON files below.
Meta JSON file
A meta.json file is the core file that contains metadata of the video for further processing either with third-party systems and software or with Cognitive Mill by running child processes, such as:
- Generating highlights with the Trailer compilation process for different sports events as the highlights sequence description.
- Manually selecting segments and creating video clips with MediaMill™.
At first, the video is processed to collect the required metadata for the following human augmentation or fully automated post-editing and highlight generation.
The meta.json file you get after running the Soccer meta process is essential for Automated Sports Highlight Generation.
The file contains data with labeled segments for further processing and metadata that contains the information about modules and module versions for internal use.
Data. Displays Segments with the following properties:
- Type. The type of content.
- Start. The start time of the segment. Ms stands for milliseconds and shows when the episode appears in the video timeline.
- End. The end time of the segment. Ms stands for milliseconds and shows when the episode appears in the video timeline.
- Label. Special markers for sorting segments according to their content, such as Goal zone, Fan zone, Full height, Whole field, Close-up (face), Side content, etc.
Some segments may have additional parameters for internal usage, such as: anomaly_score, dynamics_score, dynamics_type, median_area_person, median_person_quantity_for_frame, normalised_dynamics_score.
Segments don’t necessarily have all of the properties mentioned above.
There are two layers of segments:
1. Upper layer — type Shot. The upper layer is represented by operator’s shots, the biggest type of segments.
2. Inner layer:
- Type Subshot. The inner layer is usually represented by subshots that split operator’s shots into smaller completed parts.
- Other inner layer segment types. The types may vary depending on your content and our released version. According to the visual content, the inner layer can also be represented by other specific types, like Corner kicks, Penalty kicks, and others.
Below is a snippet of a Soccer meta.json file.
{
"data": [
{
"id": "meta",
"segments": [
{
"anomaly_score": 1,
"dynamics_score": 0,
"end": {
"ms": 5950,
"time": "0:00:05.950000"
},
"label": "Side content",
"median_area_person": 0.603,
"median_person_quantity_for_frame": 1,
"normalised_dynamics_score": 0,
"repr_ms": 2833.33,
"segments": [
{
"end": {
"ms": 5950,
"time": "0:00:05.950000"
},
"start": {
"ms": 0,
"time": "0:00:00"
},
"type": "subshot"
}
],
"start": {
"ms": 0,
"time": "0:00:00"
},
"type": "shot"
}
],
"type": "time segments"
}
],
"metadata": {
...
},
"type": "meta",
"version": "1.0.0.0"
}
Trailer JSON file
After you get the meta.json file for the video, you can also generate a trailer.json file by running a soccer trailer compilation process.
This is a child process that cannot be run without the parent soccer meta process.
A trailer.json file describes the sequence of shots and subshots of a video for different types of highlights that the system has sorted.
The file contains the following information:
- ID is the type of a trailer preset: High drama, Operator based, Tense, Mid, or Attacks.
- Start and End time properties given in milliseconds.
Additional properties that are displayed for specific trailer presets: - Type. Some presets may also display the segment’s types: shot or subshot.
- Label. Some presets, such as Operator-based and Tense, for example, may have labels: Goal zone, Fan zone, Full height, etc.
Below is a snippet of the High drama preset metadata.
{
"id": "high drama",
"segments": [
{
"end": {
"ms": 196216.66499999998,
"time": "0:03:16.216665"
},
"repr_ms": 195500,
"segments": [
{
"end": {
"ms": 196216.66499999998,
"time": "0:03:16.216665"
},
"start": {
"ms": 194950,
"time": "0:03:14.950000"
}
}
],
"start": {
"ms": 194950,
"time": "0:03:14.950000"
}
},
{
"end": {
"ms": 213550,
"time": "0:03:33.550000"
},
"repr_ms": 212466.67,
"segments": [
{
"end": {
"ms": 213550,
"time": "0:03:33.550000"
},
"start": {
"ms": 212016.66499999998,
"time": "0:03:32.016665"
}
}
]
...
This trailer.json file may be used by a MediaMill™ child process to create a trailer of your sports game.
Demo case
This guide shows you how to create Soccer Highlights via the UI at run.cognitivemill.com.
Before creating your highlight video, you need to sign in to the demo account and create your account if it’s your first visit; or sign in to your account if you already have one.
Make sure you have the required quotas to get your video processed.
To check quotas, click the name of your account in the top-right corner of the screen. You land on the My Account page with the list of your quotas.
New users are provided with trial quotas. If you don’t have the required quotas, contact us at support@aihunters.com to get them.
Now you're all set.
1. Click the Run Process button on the top navigation bar.
The Run a New Process page opens.

2. Select soccer meta from the drop-down list of the Process type field.
3. In the Title field, enter a name for your process.
4. In the File source field, either paste a link to the video you want to process — the default File link option — or click the field > select Upload file > click Select file > pick the file from your device to upload.
5. Click the Run process button.
The processing of the video has started. You can follow the progress on the Process List page, where your process appears in the current status.
When the status changes to Completed, you can:
- Download metadata for third-party systems and software.
- Review metadata and create highlight reels in the semi-automated mode in our visualizer.
- Create highlight reels by our AI in the fully automated mode.
Download metadata
After the video is processed and its status changes to Completed, you can download metadata for the processed video.
To download metadata:
1. On the Process List page, click the three vertical dots icon next to the process.
2. In the pop-up menu that appears, click Download result.

The meta.json file has been downloaded to your device.
Create highlights in fully automated mode
To create highlight reels with our AI in the fully automated mode:
1. On the Process List page, click the three vertical dots icon next to the process.
2. Click Get Set of Trailers in the pop-up menu that appears to generate a trailer.json file for creating highlight reels in the fully automated mode with MediaMill™ further on.

The Get Set of Trailers pop-up window opens.
3. Enter a name for your process in the Title field.
4. Click Generate.
By clicking the Generate button, you start a new child process that generates a trailer.json file with metadata for further creation of several popular trailer types: a high-drama preset, operator-based preset, attacks-only preset, and compressed game preset (aka mid).
Trailer Types:
- High-drama preset: gathers the most dramatic moments of the match.
- Operator-based preset: includes the most notable moments of the game based on the view of a sports director and cameraman.
- Attacks-only preset (high compressed game): shows a summarized game with attacks only.
- Compressed game preset (mid compressed game): includes the game itself without breaks, interviews, ads, etc. No game action moments are missing.
The process has been created.
You land on the Process List page.
5. Click the title of the process for which you’ve just created a trailer compilation process.
The Cognitive Mill visualizer opens.
The process appears at the bottom of the page in the current status.
6. When the status of the process changes to Completed, you can:
- Download the trailer metadata to trim the trailer sequences with third-party systems and software.
- Generate highlight reels with MediaMill™.
Download trailer metadata

1. To download the trailer metadata, click the three vertical dots icon next to the process.
2. In the pop-up menu that appears, click Download results, then trailer.json and/or operator.json.
The trailer.json file contains all available presents, except for the operator-based preset, which is in the separate operator.json file.
The files get downloaded to your device.
Generate highlight reels
To generate highlight reels:
1. Click Run MediaMill above the video in the visualizer.

The MediaMill page opens.

2. Enter a Title for your media generation process.
3. Click the Source field to open the drop-down list and select the name of your trailer compilation process.
4. Select your Presets. You can pick one to five presets at a time.
5. Click Run Process.
The preparation of your media file has been successfully started. The process appears at the bottom of the page.
When the status of the process changes to Completed, click the three vertical dots icon, and then, in the pop-up menu that opens, click Get [preset name] media (Get high drama media, for example) to open and preview the reel in a separate tab.

The reel opens in a new tab.
To download the reel, click the three vertical dots icon in the bottom-right corner, and then click Download.
The media file has been downloaded to your device.
Alternatively, you can select Editor in the Source drop-down list, and create highlight reels in the semi-automated mode using pre-generated trailer sequences.
Create highlights in semi-automated mode
To review metadata and create highlight reels in the semi-automated mode in the visualizer:
1. On the Process List page, click the title of the parent process.
The process opens in the Cognitive Mill visualizer.
2. In Meta on the right-side panel, view the color and name for each element, and select the items you want to preview. Each active layer of metadata is presented as a double-view timeline (overall and zoomed-in).

You can:
- Include all the Meta items (all meta is selected by default) by clicking Add to Editor above the meta timeline under the video.
- Manually click any of the visualized frames to add it to the Editor.
Click it again to remove it from the Editor.

3. When the selected segments are added to the Editor, optionally, click Play to preview. You can remove the unwanted ones by clicking them.
4. When ready, click Run MediaMill.

The MediaMill page opens.

5. Enter a Title for your process.
6. Leave the default Editor option selected in the Source drop-down list.
7. Click Generate.
The preparation of your media file has been successfully started. It appears at the bottom of the page.
When the status of the process changes to Completed, click the three vertical dots icon, and then click Get out media to open and preview the reel in a separate tab.
The reel opens in a new tab.
To download it, click the three vertical dots icon in the bottom-right corner, and then click Download.
The media file has been downloaded to your device.
Current challenges
- The limited number of sports covered.
Now, we process videos only for the following sports events: soccer, American football, hockey, basketball, and boxing.
But this challenge is easy to overcome. As the core principles of how the algorithms work are similar, we can customize the existing flow for any kind of sports you need. - 'No Play' scenes in certain types of trailers.
Operator-based, High drama, and Tense trailers may contain scenes that do not belong to the game action when the players are just standing or greeting each other. These cases are rare and we keep on improving our algorithms to eliminate such cases. - Rare 'static' goals may be missing in some Mid and Attacks trailers.
When the scenes before the goal is scored are quite static and lack action, the system may miss the goal and fail to identify the goal moment as an attack. Though such cases are very rare, we are working hard to reduce these cases to zero. - Some beautiful moments may be missing.
Beautiful scenes and camera angles may be ignored by the system sometimes, though they would make a trailer more spectacular, the system doesn’t identify them as core scenes. The cases are very specific and do not affect the content of the trailers.
We keep on searching for the most efficient ways how to overcome the current challenges so that in the following versions of CognitiveReelz™ Sports we won’t face them anymore.