Create video summaries

api.video's AI-driven summarization feature can generate concise and accurate outlines of your videos. Each summary consist of a title, an abstract, and key takeaways based on the exact contents of a video. All of these are available directly in the player, on your dashboard, and through the API.

Provide your audience with detailed descriptions and key points of your videos, and also enable more inclusive and accessible content by inviting deaf or hard-of-hearing users!

A summary is based on the transcription of a video. api.video uses Whisper for multilingual speech recognition and transcripton. We run Whisper's ASR models on our own infrastructure and do not expose data outside our service. You control who gets access to your videos and summaries.

Summary data structure

Summaries consist of 3 elements:

a title that clearly notes the topic of the video
an abstract that provides a short outline of the contents of the video
3 takeaways in chronologial order, highlighting the key points of the video

Each summary is identified by a summaryId. The API separates summary data into summary and source objects. A summary object contains information about the summary:

{
  "summaryId": "summary_1CGHWuXjhxmeH4WiZ51234",
  "createdAt": "2024-07-14T23:36:07+00:00",
  "updatedAt": "2024-07-14T23:36:07+00:00",
  "videoId": "vilkR8K3N7yrRcxcMt91234",
  "origin": "auto",
  "sourceStatus": "completed"
}

The source contains the actual title, abstract, and takeaways:

{
  "title": "A short lecture on quantum theory",
  "abstract": "In this lecture, we discuss how complicated quantum theory is, using the famous example of Schrödingers cat. We also discuss practical applications like quantum computing.",
  "takeaways": [
    "Quantum theory is complicated.",
    "Schrödinger's cat is neither dead, nor alive.",
    "Quantum computers are super cool."
  ]
}

Visit the API reference for detailed explanations of each field and attribute in the summary object.

How to create summaries

You have 3 methods to create summaries. You can use:

the dashboard to enable summary generation during video upload, and for already uploaded videos
the /videos endpoint for automatic summarization via the API
the /summaries endpoint for more granular control on summary creation via the API

This guide explains all 3 methods.

Generate summaries in the dashboard

When you use the dashboard to upload videos, simply switch on the Summary toggle during the upload process.

A summary is based on the transcription of a video. When you create summaries, we recommend that you also enable transcription. This enables our API to create video summaries faster.

For already updated videos, open the Video Details page and then the Summarization tab. Click on Summarize to generate a summary, or Summarize again to recreate a summary.

Automatic summarization

When you use the API to upload videos, the easiest method to enable summary creation when you create a video object.

To enable summarization, set the optional transcriptSummary parameter in your POST request to the Create video object endpoint. We recommend that you also enable the optional transcript and language parameters.

When you define the video language, the API creates a summary of the video using the language you specify. Check out the list of supported languages here. If you do not specify a language for the video, the API will detect it automatically.

Field	Type	Description
`transcriptSummary`	`boolean`	When `true`, the API generates a summary for the video. The default value is `false`.
`transcript`	`boolean`	When `true`, the API generates a transcript for the video. The default value is `false`.
`language`	`string`	A valid language identifier using IETF language tags. You can use primary subtags like `en` or `fr`. When the value in your request does not match any covered language, the API returns an error.

You can also trigger automatic summarization for already uploaded videos. Use the same parameters in your PATCH request to the Update video object endpoint.

Manual summarization

This method enables you to control every step of the summary creation. We recommend that you use this method in scenarios where you want to manually create, update, or edit the generated summary.

To summarize already uploaded videos manually, use a POST request to the Summaries endpoint. You have the option to define the origin of the summary generation with the origin request parameter.

Field	Type	Description
`origin`	`string`	Use this optional parameter to define how the API generates the summary. The only allowed value is `auto`, which means that the API generates a summary automatically.

If you do not set the origin parameter in your request, the API will not generate a summary automatically.

In this case, sourceStatus will return missing, and you have to manually add the summary source using the PATCH /summaries/{summaryId}/source endpoint operation.

Manage summaries

The API provides dedicated endpoints to list, update, and delete summaries. You can use:

the /summaries endpoint to GET a list of all summaries in your project, and to filter the list based on videoId, origin, or sourceStatus
the /summaries/{summaryId} endpoint to DELETE a summary
the /summaries/{summaryId}/source endpoint to GET the source of a specific summary, and to PATCH an existing summary.

Summary generation statuses

The summary object returns the status of summary generation in the sourceStatus field. These are the available statuses:

missing: the source for a summary is not present.
waiting : the source video is being processed and a summary will be generated.
failed: a technical issue prevented summary generation.
completed: the summary is generated.
unprocessable: the source video is unsuitable for summary generation. An example for this is a source video that has no audio, or has less than 50 words.

Using the PATCH /summaries/{summaryId}/source endpoint operation is only allowed for summaries where sourceStatus is missing. The main purpose of this endpoint is to provide a direct way to edit or update summary sources.

If you want to edit a summary where a source is already present, you first need to delete the existing summary.

Supported languages

The API creates summaries using the same language as for transcripts. Check out the Generate transcripts guide for the list of languages

Limitations

The API requires at least 50 spoken words in the video you upload to create a transcript and a summary.
The length of a summary abstract is between 20 to 300 words. The exact length of the abstract depends on the contents of the video and will be roughly 1/8th of the transcript.
When you set the language parameter, make sure that it matches the actual language used in the video. Your setting forces the API to summarize in that language. Mismatching language settings or videos with dialogue in multiple languages can return low quality summaries.

Next steps

Learn more about video transcription in the Generate transcripts guide
Check out other AI-driven features like video translation and AI summary in our blog