CognitiveSkip™

Product and Solution Overview

CognitiveSkip™ is a Cognitive Mill product that offers you a unique solution for safely skipping unessential parts of your video content like opening and closing credits, ads, etc. It enables smart EPG navigation, its automated correction, and TV ads skipping and replacement in an EPG.

Let’s review one of the CognitiveSkip™ solutions — End Credits Skip Automation — that analyzes video content and marks the so-called Safe skip zone that can be skipped without missing anything essential.

The End Credit Skip Automation solution accurately identifies where the credits start and end and you get just one end credits detection timestamp to make an informed decision.

Benefits:

  • You get the Safe point, after which you can safely skip the credits.
  • You won’t miss post- and in-credit scenes, anything important like extra scenes, bloopers, or creatively designed credits that can be a part of the story.
    Such credits will be identified as standout credits and won’t be marked for safe skipping so that you don’t miss anything important and decide yourself to skip them or not.
  • You can interact with the platform either via the UI or via the API.
  • You paste the video URL and hit the button — the rest is done automatically.
  • You get a Skip meta JSON file to process it with third-party systems and software.

Output Metadata for End Credits Skip Automation

A meta.json file is the core file that contains metadata of the video for further processing either with third-party systems and software or with Cognitive Mill by running child processes, like generating highlights with MediaMill™ and trailer compilation for different sports events.

The result.json file you get after running a Skip meta process is essential for End Credits Skip Automation. It contains the following metadata:

  • Segments. Display the start time and end time of the content item and its type.
  • Time markers. Display the key timestamps in the segments.

Terminology

  • Safe skip is a segment in the video timeline that can be skipped without the risk of missing something important. It is also known as Safe skip zone or just Skip zone.
  • Safe point is the final timestamp after which nothing essential can take place, the following content, if any, can be safely skipped or even cut. This is the point when you can switch your audience to the next movie, series, TV show, etc.
  • Closing credits are a long sheet of text with credits only, without anything belonging to the main action. It can be safely skipped without missing the essential.
    Closing credits can be included in the Safe skip segment, or the beginning of closing credits can be the Safe point if nothing important follows them.
  • Standout end credits are the credits that are either creatively designed or include anything important like extra scenes, bloopers, or appear while the action in the background still goes on. We identify this type of credits and mark it. It is never included in the Safe skip so that you don’t miss anything important.
  • Post-credits are movie companies’ logos usually displayed after closing credits. Though they get into the Safe skip zone, we identify and mark separately for your convenience.

Video Timeline Structure

For a better understanding of what is included in the result.json file, let’s review an example of a movie timeline structure with standout end credits and closing credits followed by a post-credits scene and see how it is shown in a meta.json file.

Above is the timeline structure of a movie that includes:

Standout end credits. Credits that are either creatively designed or include anything important like extra scenes, bloopers, etc. We identify this type of credits and mark it. It is never included in the Safe skip so that you don’t miss anything important.

Closing credits. The long sheet of text with credits that can be safely skipped without missing anything essential. This is the Safe skip in this example but they do not always coincide.

Post-credits scene. A scene that follows the closing credits and belongs to the action part. The system identifies it as an action so we guarantee that nothing important will be missed.

Safe point. The final timestamp after which nothing essential can take place, and you can switch your audience to the next movie, series, TV show, etc.

Skip Meta JSON File

Let’s now review the result.json file that contains metadata of the movie with the above-described pattern.

For Segments, the following data is displayed:

  • End. The end time of the segment. Ms stands for milliseconds, and time shows when the episode appears in the video timeline.
  • Start. The start time of the segment. Ms stands for milliseconds, and time shows when the episode appears in the video timeline.
  • Type. The type of content can be the following: closing credits, standout end credits, post credits, and safe skip.

For Time markers, the following data is displayed:

  • Ms and time. The key points in the video timeline for automatic skipping.
  • Type. The type of content can be the following: closing credits, standout end credits, post credits, and safe point.
{
  "segments": [
    {
      "end": {
        "ms": 5450000,
        "time": "1:30:50"
      },
      "start": {
        "ms": 5400000,
        "time": "1:30:00"
      },
      "type": "standout end credits"
    },
    {
      "end": {
        "ms": 5570000,
        "time": "1:32:50"
      },
      "start": {
        "ms": 5450000,
        "time": "1:30:50"
      },
      "type": "closing credits"
    },
    {
      "end": {
        "ms": 5570000,
        "time": "1:32:50"
      },
      "start": {
        "ms": 5450000,
        "time": "1:30:50"
      },
      "type": "safe skip"
    }
  ],
  "time_markers": [
    {
      "ms": 5400000,
      "time": "1:30:00",
      "type": "standout end credits"
    },
    {
      "ms": 5450000,
      "time": "1:30:50",
      "type": "closing credits"
    },
    {
      "ms": 5630000,
      "time": "1:34:20",
      "type": "safe point"
    }
  ]
}

Segments in the example above include end and start time for two content types:

  • Standout end credits that we identify but leave the final decision to skip them or not to you.
  • Closing credits are the credits with nothing to miss except for the long sheet of credits text, for which we guarantee that you can skip them.
  • Safe skip is a part that can be skipped without the risk of missing something important.
    In our example, the Closing credits and Safe skip belong to the same segment.

Time markers in the example above include timestamps for three content types:

  • Standout end credits. This timestamp marks the start time of the standout end credits.
  • Closing credits. This timestamp marks the start time of the closing credits, the beginning of the Safe skip zone, in our case.
  • Safe point. This timestamp marks the end of the post-credits scene, and the following content, if any, can be safely skipped or even cut.

Other Cases

1. A Safe point can also be at the beginning of Closing credits if nothing essential comes after them, as shown in the timeline structure below.

After you process a video that has such structure, you get a result.json file similar to the one displayed below.

{
  "segments": [
    {
      "end": {
        "ms": 881920,
        "time": "0:14:41.920000"
      },
      "start": {
        "ms": 876160,
        "time": "0:14:36.160000",
        "type": "safe point"
      }
    }
  ]
}

2. In some cases, a Safe skip zone may not coincide with Closing credits.
It happens, for example, when Closing credits start, then get interrupted by an in-credits scene, then they continue.

The system cannot break up the Safe skip zone into parts. There can be only one Safe skip per video. The In-credits scene gets identified as an action that cannot be skipped, so the Safe skip starts after the In-credits scene ends.

{
  "segments": [
    {
      "segments": [
        {
          "end": {
            "ms": 5450000,
            "time": "1:30:50"
          },
          "start": {
            "ms": 5400000,
            "time": "1:30:00"
          },
          "type": "standout end credits"
        },
        {
          "end": {
            "ms": 5570000,
            "time": "1:32:50"
          },
          "start": {
            "ms": 5450000,
            "time": "1:30:50"
          },
          "type": "closing credits"
        },
        {
          "end": {
            "ms": 5570000,
            "time": "1:32:50"
          },
          "start": {
            "ms": 5500000,
            "time": "1:31:40"
          },
          "type": "safe skip"
        }
      ],
    }
  ],
  "time_markers": [
    {
      "ms": 5400000,
      "time": "1:30:00",
      "type": "standout end credits"
    },
    {
      "ms": 5450000,
      "time": "1:30:50",
      "type": "closing credits"
    },
    {
      "ms": 5630000,
      "time": "1:34:20",
      "type": "safe point"
    }
  ]
}

The system is smart enough to analyze the context and recognize video content patterns. But to avoid extra information, the generated result.json file contains only the core data essential for solving your business tasks — the Safe skip timeframe and the Safe point timestamp.

Demo Case - Get Skip Meta JSON File via UI

This guide will show you how to run the Skip meta process to get a result.json file via the UI.

1. Sign in to your account at run.cognitivemill.com.
If you don’t have an account, please sign in to the demo account and create your own account.

2. Check that you have the Skip meta quota to run the Skip meta process, if not — contact us at support@aihunters.com to get it.

3. Click the Run process button.

The Run a New Process page opens.

4. Select Skip meta from the Process type drop-down list.
Note that only the types of processes you have quotas for are available here.

5. In the Title field, enter a name for your process.

6. In the Video URL field, paste the link to the video you want to get the result.json file for.

7. (Optional) Enter the Start time and End time (in milliseconds) to specify the time interval that the system should analyze.

8. Hit Run process.
Your newly created process appears at the top of the Process List in the current status.

When the status changes to Completed, you can download metadata for third-party systems and software.

9. On the Process List page, click the three vertical dots icon next to the process.

10. In the pop-up menu that appears, click Get result.json.
The result.json file is downloaded to your device.

You can also view how it works in the Cognitive Mill visualizer.
Click the title of the process to open the video in the visualizer.

The visualizer opens with Markers and Segments selected in Settings by default and marked in the timelines.
Here you can see the Safe point, after which the closing credits start and safe skipping is guaranteed.

Current Challenges

  • One Safe skip per video.
    Only one Safe skip zone can be identified in a video now. If Closing credits are broken up into two or more segments by mid-credits scenes and followed by a post-credits scene, only the last part of such split Closing credits will be included in the Safe skip zone. For a more detailed example, view Case 2.
  • Standout end credits.
    Standout end credits are sometimes hard to identify because of the variety of styles and forms they can take. In complicated cases, when Standout end credits cannot be accurately recognized, the system doesn’t include them in the skip meta JSON file. It doesn’t affect the End credits skip automation but helps to prevent skipping something important.
  • Animated logos.
    Movie studio animated logos that sometimes appear at the end of closing credits may also be confusing for the system and mistakenly recognized as the main content. In this case, the system will put the Safe point marker after the animated logo, and the Safe skip zone will be before the logo, covering the closing credits.
  • Indistinct credits.
    In some cases, when the action still goes on, or the movie scenes change, the appearing credits are indistinct and not clearly seen. The system may ignore such credits and not include them in the meta.json file.

We are working hard on the most efficient ways how to overcome the current challenges so that in the following versions of CognitiveSkip™ we won’t face them anymore.


We use cookies to ensure that we give you the best experience on our website. Read cookies policies.

Loading