Video summarization as the name implies, is a short summary of the content of a longer video The purpose of video summarization

is to extract from a video, a limited number of key frames that convey the meaning of the whole video at a glance. This project proposes to formulate video summarization as a search problem and to use Genetic Algorithms as the search algorithm.

Genetic algorithms belong to the class of evolutionary algorithms (EA), which generate solutions for problems using techniques inspired by natural evolution, such as inheritance, mutation, selection, and crossover.

An original video consists of the following components- Scenes, Shots and frames, in a hierarchical order
The original video is split up into scenes and then to shots which finally give frames.

In the video summarization process, only the frames are considered. A group of selected frames are combined together to form candidate summaries.

Original Video

U s e r

Scenes

Candidate Summaries

Shots

Frames

Summarized Video Selection/Crossover/Mutation

 This

forms the first step of the video summarization process where the whole video is split up into frames

Defining the population size (Candidate Summaries) User analysis of the generated summaries by giving a score to each of them Generating the next set of summaries through the GA procedures of crossover and mutation, based on the user given scores Iterating the process until a preferable summary is obtained

STEPS INVOLVED

A text field is associated with each of the candidate summaries where the user can rate the summary by giving it a score of his choice. It is assumed that the one with the highest score forms the most optimal summary. This experimentation was carried out with four candidate summaries. Four corresponding scores are given for each of the four candidate summaries. The algorithm proceeds in such a way that top three scores are taken and the summary with the highest score is crossed over and mutated with the other two to form a new set of candidate summaries.

The GA proceeds through the following stages

Selection Crossover Mutation

The population size is fixed as 4 Based on the user scores for each of the summaries, the highest 3 scores are selected and the last one is ignored The crossover and mutation operations are performed based on these 3 scores to generate the next set of summaries The mutation and crossover probabilities are randomly generated and a threshold is set, above which mutation and crossover happens

Once the scores are generated, the summaries are processed by the genetic algorithm based on the following two factors: Crossover Mutation

Crossover: Crossover is a genetic operator

It is analogous to reproduction and biological crossover, upon which genetic algorithms are based. The first and the second summaries are taken and they are combined together so that the frames of both the summaries are arranged in the temporal order. Two new summaries are generated from this combination such that the new summaries will have the alternate values of the combination, i.e. the first value of the combination goes to the first summary, and second value goes to the second summary and so on.

Summary 1

84 32 117

128 117 128

199 176 176

297 312 199 297 312

Summary 2
32 84

New Summary 1 New Summary 2

32
84

117
128

176
199

297
312

Mutation: Mutation is a genetic operator used to maintain genetic diversity from one generation of a population of algorithm chromosomes to the next. It is analogous to biological mutation.

The purpose of mutation in GAs is to prevent the population of chromosomes from becoming too similar to each other, thus slowing or even stopping evolution. This experiment implements mutation after the crossover. After the generation of a new set of summaries after crossover, the elements in it are mutated, i.e. values within are replaced by a new set of values and are arranged in the temporal order to preserve the semantics.

Applications of video summarization

Video surveillance: In surveillance video, instead of viewing the whole video summary of video can be of use. Internet: In internet, before downloading a full video, download the summary video and check if the video is relevant or not in that case download the video In movie trailers In cricket highlights etc

THANK YOU