CognitiveReelz™. Sports

Product and Solution Overview

CognitiveReelz™ is one of the Cognitive Mill products that offers you an outstanding solution for creating highlight reels for different sports events, movies, TV shows, streamings, etc. It also analyzes the pattern of a video, summarizes video content, creates trailers and highlights, and more.

Our cognitive computing cloud solution watches and analyzes video content, and then selects highlights as a human would, and does it much faster. It takes a few clicks to start the processing and the machine does the rest. You can also select the types of highlights you want to include in the trailer before it is generated.

One of the core CognitiveReelz™ solutions is Automated Sports Highlight Generation. This solution saves you much time, effort, and money by automatically creating highlight reels of a sports event both live and recorded. The highlights of a live event will be ready in seconds after it is over.

Now the AI-powered cloud ‘robot’ is able to analyze and process the following events:

  • Soccer matches
  • American football matches
  • Basketball matches
  • Hockey matches
  • Boxing fights.

The system analyzes video content, cuts the video into multi-level pieces, and then sorts and labels them according to the content type.

It recognizes and sorts separately game action and side content. In the game action part, the system can identify tense, high-drama moments, attacks, and etc., that can be used to generate the corresponding types of trailers.

You need to run a parent process to get output metadata first. Depending on the sports event, you should select the corresponding type of process: Soccer meta, American football meta, Basketball meta, Hockey meta, or Boxing meta.

After running a parent process, you can get a meta.json file that contains all the required metadata for further processing with MediaMill™ or with third-party software, or you can run a child Trailer compilation process and get a trailer.json file with metadata for different types of trailers sorted and labeled.

Benefits:

  • You get metadata for creating highlight reels of a sports event in JSON format.
  • You can generate a trailer.json file for further creation of a number of popular types of trailers.
  • You can create highlights in the fully or semi-automated mode using the Cognitive Mill child processes or with the help of third-party systems and software.
  • You can process both recorded videos and live streams.
  • The video is processed in the fully automated mode without human involvement, which is faster and cheaper.
  • You can interact with the platform either via the UI or via the API.
  • We offer ready-made solutions for Soccer, American football, Hockey, Basketball, and Boxing. But our system is flexible and smart enough to adapt to other sports and you can request any other kind of sports according to your business needs.

Let’s review what is included in the output meta.json file after running the metadata analysis.

Output Metadata for Automated Sports Highlight Generation

We’ve taken the analysis of a Soccer match as an example, but all other kinds of sports are analyzed the same way. And the output metadata you get is similar to the displayed in the JSON files below.

Meta JSON File

A meta.json file is the core file that contains metadata of the video for further processing either with third-party systems and software or with Cognitive Mill by running child processes, such as:

- Generating highlights with the Trailer compilation process for different sports events as the highlights sequence description.

- Manually selecting segments and creating video clips with MediaMill™.

At first, the video is processed to collect the required metadata for the following human augmentation or fully automated post-editing and highlight generation.

To facilitate manual trailer creation in semi-automated mode:

1. Run the Soccer meta collection parent process.

2. Get meta.json.

3. Manually select the segments that should be included in the highlights generated with the Cognitive Mill web editing software or external software.

To generate a trailer in fully automated mode:

1. Run the Soccer meta collection parent process.

2. Get meta.json.

3. Soccer Trailer compilation child process.

4. Get trailer.json.

Both modes use the MediaMill™ child process to create result videos.

Now, let’s review the metadata that you get after running a Soccer meta process.

The meta.json file you get after running a Soccer meta process is essential for Automated Sports Highlight Generation.

The file displays Segments with the following properties::

  • Type. The type of content.
  • Start. The start time of the segment. Ms stands for milliseconds and shows when the episode appears in the video timeline.
  • End. The end time of the segment. Ms stands for milliseconds and shows when the episode appears in the video timeline.
  • Label. Special markers for sorting segments according to their content, such as Goal zone, Fan zone, Full height, Whole field, Close-up (face), Side content, etc.

Some segments may have additional parameters for internal usage, such as:anomaly_score, dynamics_score, dynamics_type, median_area_person, median_person_quantity_for_frame, normalised_dynamics_score.

Segments don’t necessarily have all of the properties mentioned above.

There are two layers of segments:

1. Upper layer — type Shot. The upper layer is represented by operator’s shots, the biggest type of segments.

2. Inner layer:

  • Type Subshot. The inner layer is usually represented by subshots that split operator’s shots into smaller completed parts.
  • Other Inner layers segment types. The types may vary depending on your content and our released version. According to the visual content, the inner layer can also be represented by other specific types, like Corner kicks, Penalty kicks, and others.

Below is a piece of a Soccer meta.json file.

{
  "segments": [
    {
      "anomaly_score": 0.9997883904891389,
      "dynamics_score": 0.903,
      "dynamics_type": "static",
      "end": {
        "ms": 10560
      },
      "label": "Side content",
      "median_area_person": 0.023,
      "median_person_quantity_for_frame": 3,
      "normalised_dynamics_score": 0.08551136363636364,
      "segments": [
        {
          "end": {
            "ms": 2280
          },
          "start": {
            "ms": 0
          },
          "type": "subshot"
        },
        {
          "end": {
            "ms": 4880
          },
          "start": {
            "ms": 2320
          },
          "type": "subshot"
        },
        {
          "end": {
            "ms": 7120
          },
          "start": {
            "ms": 4920
          },
          "type": "subshot"
        },
        {
          "end": {
            "ms": 10560
          },
          "start": {
            "ms": 7160
          },
          "type": "subshot"
        }
      ],
      "start": {
        "ms": 0
      },
      "type": "shot"
    },
    {
      "anomaly_score": 1,
      "dynamics_score": 0.771,
      "dynamics_type": "slow motion",
      "end": {
        "ms": 14120
      },
      "label": "Side content",
      "normalised_dynamics_score": 0.21903409090909093,
      "segments": [
        {
          "end": {
            "ms": 14120
          },
          "start": {
            "ms": 10600
          },
          "type": "subshot"
        },
        {
          "dynamics_score": 0,
          "end": {
            "ms": 14120
          },
          "label": "Whole field",
          "start": {
            "ms": 10600
          },
          "type": "subshot"
        }
      ],

The labels in the Soccer meta.json file correspond to the Meta options in Settings> Timelines of the Cognitive Mill visualizer.

Trailer JSON File

After you get a meta.json file for the video, you can also generate a trailer.json file by running a Soccer trailer compilation process.

This is a child process that cannot be run without the parent Soccer meta process.

A trailer.json file describes the sequence of shots and subshots of a video for different types of highlights that the system has sorted.

The file contains the following information:

  • ID is the type of a trailer preset: High drama, Operator based, Tense, Mid, or Attacks.
  • Start and End time properties given in milliseconds.
    Additional properties that are displayed for specific trailer presets:
  • Type. Some presets may also display the segment’s types: shot or subshot.
  • Label. Some presets, such as Operator-based and Tense, for example, may have labels: Goal zone, Fan zone, Full height, etc.

Below is a piece of High drama preset metadata.

{
  "id": "high drama",
  "segments": [
    {
      "end": {
        "ms": 361160
      },
      "start": {
        "ms": 357720
      },
      "type": "shot"
    },
    {
      "end": {
        "ms": 235240
      },
      "start": {
        "ms": 233480
      },
      "type": "shot"
    },

This trailer.json file may be used by a MediaMill™ child process to create a trailer of your sports game.

Demo Case - Soccer Highlights via UI

This guide will show you how you can create Soccer Highlights via the UI at run.cognitivemill.com.

Before creating your highlight video, you need to sign in to the demo account and create your account if it’s your first visit; or sign in to your account if you already have one. After that, you should make sure you have the required quotas to get your video processed.

New users are provided with trial quotas. If you don’t have the required quotas, contact us at support@aihunters.com to get them.

When you are all set, follow the instructions:

1. Click Run Process.

The Run a New Process page opens.

Note that you can run only those processes for which you have quotas.

2. Select Soccer meta from the drop-down list of the Process type field.

3. In the Title field, enter a name for your highlight parent process.

4. In the Video URL field, paste the link to the video you want to get the highlights for.

5. Click the Run process button.

The processing of the video has started. You can follow the progress on the Process List page, where your process appears in the current status.

6. When the status changes to Completed, you can:

Download Metadata

After the video is processed and its status changes to Completed, you can download metadata for the processed video.

To download metadata:

1. On the Process List page, click the three vertical dots next to the parent process.

2. In the pop-up menu that appears, click Get meta.json.

The meta.json file has been downloaded to your device.

Create Highlights in Fully Automated Mode

To create highlight reels with our AI in the fully automated mode:

1. On the Process List page, click the three vertical dots next to the parent process.

2. Click Get Set of Trailers in the pop-up menu that appears to generate highlight reels in the fully automated mode. The Get Set of Trailers pop-up window opens.

3. Click Generate.

By clicking the Generate button, you create a new child process that simultaneously generates several popular trailer types: a high-drama preset, operator-based preset, attacks-only preset, and compressed game preset.

Trailer Types:

  • High-drama preset: gathers the most dramatic moments of the match.
  • Operator-based preset: includes the most notable moments of the game based on the view of a sports director and cameraman.
  • Attacks-only preset (high compressed game): shows a summarized game with attacks only.
  • Compressed game preset (mid compressed game): includes the game itself without breaks, interviews, ads, etc. No game action moments are missing.

The set of trailers has been generated.

4. Close the Get Set of Trailers pop-up window. You land on the Process List page.

5. Click the title of the process for which you’ve just generated trailers.

The Cognitive Mill visualizer opens.

6. When the status of the subprocess changes to Completed, you can:

  • Download the trailer metadata to trim the trailer sequences with third-party systems and software.
  • Review the trailer in our visualizer, and generate media files for the trailers.

Download Trailer Metadata

1. To download the trailer metadata, click the three vertical dots next to the child process.

2. In the pop-up menu that appears, click Get trailer.json.

The trailer.json file has been downloaded to your device.

Review Trailer and Generate Media Files

To review the trailer in our visualizer and generate media files for the trailers:

1. Click Run Media Mill.

The Soccer meta overlay opens.

2. Enter a Title for your media.

3. Select the Source drop-down list.

4. Select your Presets. You can pick one to five presets at a time.

5. Click Generate.

The preparation of your media file has been successfully started. It appears at the bottom of the page.

When the status of the process has changed to Completed, click the three vertical dots icon, and then click Get output media to open and preview the trailer in a separate tab.

The trailer opens in a new tab.
To download the trailer, click the three vertical dots icon in the bottom-right corner, and then click Download.

The media file has been downloaded to your device.

Alternatively, you can select Editor in the Source drop-down list, and create highlight reels in the semi-automated mode using pre-generated trailer sequences.

Create Highlights in Semi-Automated Mode

To review metadata and create highlight reels in the semi-automated mode in our visualizer:

1. On the Process List page, click the title of the parent process.

The process is opened in the Cognitive Mill visualizer.

2. Go to Settings > Timelines to view the color and name for each element, and select the items you want to preview. Each active layer of metadata is presented as a double-view timeline (overall and zoomed-in).

You can:

  • Include all the Types and Timeline items selected in Settings by clicking Add to Editor.
  • Manually click any element of the visualized fields to add it to the Editor.
    Click it again to remove it from the Editor.

3. When the selected segments are added to the Editor, you can quickly browse through the items and remove the unwanted ones by clicking them.

4. When ready, click Run Media Mill.

The Soccer meta overlay opens.

5. Enter a Title for your media.

6. Select Editor from the Source drop-down list.

7. Click Generate.

The preparation of your media file has been successfully started. It appears at the bottom of the page.

When the status of the process has changed to Completed, click the three vertical dots icon, and then click Get output media to open and preview the trailer in a separate tab.

The trailer opens in a new tab.
To download the trailer, click the three vertical dots icon in the bottom-right corner, and then click Download.

The media file has been downloaded to your device.

Current Challenges

  • The limited number of sports covered.
    Now, we process videos only for the following sports events: soccer, American football, hockey, basketball, and boxing.
    But this challenge is easy to overcome. As the core principles of how the algorithms work are similar, we can customize the existing flow for any kind of sports you need.
  • 'No Play' scenes in certain types of trailers.
    Operator-based, High drama, and Tense trailers may contain scenes that do not belong to the game action when the players are just standing or greeting each other. These cases are rare and we keep on improving our algorithms to eliminate such cases.
  • Rare 'static' goals may be missing in some Mid and Attacks trailers.
    When the scenes before the goal is scored are quite static and lack action, the system may miss the goal and fail to identify the goal moment as an attack. Though such cases are very rare, we are working hard to reduce these cases to zero.
  • Some beautiful moments may be missing.
    Beautiful scenes and camera angles may be ignored by the system sometimes, though they would make a trailer more spectacular, the system doesn’t identify them as core scenes. The cases are very specific and do not affect the content of the trailers.

We keep on searching for the most efficient ways how to overcome the current challenges so that in the following versions of CognitiveReelz™ we won’t face them anymore.


We use cookies to ensure that we give you the best experience on our website. Read cookies policies.

Loading