CognitiveShapes™

Product and solution overview

CognitiveShapes™ is a Cognitive Mill product that offers you a top-notch Graphics Detection AI service that finds repetitive moving graphics, such as animated brand and TV logos, ad insertions in the TV and sports media events.

The AI-powered cloud robot identifies repetitive opening and closing animated graphics in any type of content, such as the news, TV shows, series, sports events, etc.

It detects game replays, ad insertions, other side content enclosed in animated graphics in both live and recorded videos and puts the corresponding time markers onto the video timeline for your convenient video post-processing.

CognitiveShapes™ offers a fast and reliable solution with high performance and a 100 percent guarantee against false detection.

We apply geometrical assessment and computer vision shapes and dynamics analysis where neural networks don’t work well enough. The graphics animation descriptor is color-agnostic and is based on the unique sequence of shapes.

The flow doesn’t need training and supervision procedures, which significantly reduces your time to market.

You provide us with an example of the animated graphics image you need to detect — our team creates a graphics animation descriptor or a graphics animation preset if you need to analyze multiple animated graphics. The preset is prepared in a couple of hours, and you can get it from us as a file, or we can store it in our system under your ID.

After that, you can run graphics meta during live streams or process recorded video files, and the service will identify the exact moments where the animated graphics sequence appears.

And this is only a part of what our Graphics Detection solution can do.

After running the parent graphics meta process, you get a result.json file that contains the names of the graphics animation descriptor, or masks, and their time markers. You can use the data for further integrations via the API and processing with third-party software.

You can also preview the processed video with time markers in the Cognitive Mill visualizer.

Benefits:

• CognitiveShapes™ is a highly efficient service for live streams. Originally tailored for live streams, it works fast and accurately. But it’s also suitable for recorded video content.
• By applying the geometrical assessment and computer vision, we have achieved 100 percent protection against false detection.
• CognitiveShapes™ automatically processes any type of content. It can detect repetitive moving graphics in sports events, movies, series, TV shows, or news.
• A fast and easy start.
• After running the graphics meta process, you get the JSON file with time markers of the required graphics animation.
• The system supports the analysis of opening and closing graphics animation. Contact us to implement this feature.

Output metadata

The result.json file you get after running the graphics meta process contains the following information:

Data. includes:

• Events. The time markers for each moving graphics item (mask) according to your sample that appears in the video.
• Id. The entity name.
• Type. The entity format type.

Metadata. Contains the information about the modules and versions for internal use.

Below is a snippet of a result.json file.

  {
    "data": [
      {
        "events": [
          {
            "mask_name": "laliga_union_2603",
            "ms": 275360,
            "time": "0:04:35.360000"
          },
          {
            "mask_name": "laliga_union_2603",
            "ms": 282160,
            "time": "0:04:42.160000"
          },
          {
            "mask_name": "laliga_union_2603",
            "ms": 509320,
            "time": "0:08:29.320000"
          },
          {
            "mask_name": "laliga_union_2603",
            "ms": 513840,
            "time": "0:08:33.840000"
          }
        ],
        "id": "graphics events",
        "type": "time events"
      }
    ],
    "metadata": {
      ...
    }

  

The result.json file contains events, the time markers that display the following data:

• Mask_name. The name of the animation you provided in the graphics animation preset.
• Ms. Stands for milliseconds and time — for the precise moment in the timeline when the required graphics image appears.

Demo case

This guide shows you how to generate the graphics meta JSON file for further integration and processing via the UI at run.cognitivemill.com.

Before processing your video:

1. Sign in to your account or register if it’s your first visit.

2. Make sure you have the mask ID and the graphics meta quota to get your video processed.

Check your quota in My Account by clicking the name of your account on the top navigation bar.

New users are provided with trial quotas. If you don’t have the required quotas or you haven't received the mask ID, contact us at support@aihunters.com to get them.

Now you are all set.

1. Click Run Process on the top navigation bar.

The Run a New Process page opens.

2. Select graphics meta from the drop-down list of the Process type field.
Note that you can select only those processes for which you have quotas.

3. In the Title field, enter a name for your process.

4. In the File source field, either paste a link to the video you want to process — the default File link option — or click the field > select Upload file > click Select file > pick the file from your device to upload.

5. In the Mask Id field, paste the mask ID we've prepared for you.

6. Click the Run Process button.
The processing of the video starts. You can follow the progress on the Process List page, where your process appears in the current status.

When the status of the process changes to Completed, download the result.json file with all the time markers and view the result in the Cognitive Mill visualizer.

To download the JSON file:

1. Click the three vertical dots icon.

2. In the pop-up menu that appears, click Download result.

The result.json file has been downloaded to your device.

To view the result in the Cognitive Mill visualizer:

1. Click the title of the process on the Processes page.
The Cognitive Mill visualizer opens.

2. Navigate to the time markers in the graphics events timeline to view how accurately they mark the moving graphics appearing in the video.

Current challenges

• Changes in aspect ratio can make the descriptor invalid.
  Though the system is color-agnostic and insensitive to aspect ratio resize, significant changes in aspect ratio in the processed media, compared to the original, can make the graphics animation descriptor invalid, and it will need an update.
• Frame rate changes may affect detection.
  Significant changes in frame rate may break the animation sequence, and this way can affect the detection.
• Transparent overlays may not be detected.
  If the overlays are too transparent and the background breaks the initial shapes seriously, they may not be detected.
  But anyway, we guarantee that false detection is excluded.

We are working hard on the most efficient ways to overcome the current challenges so that in the following versions of CognitiveShapes™ we won’t face them anymore.