Ad Detection and Removal Tool / Video Player

April 2013|Tags: java, programming

In my Multimedia Systems Design class, we were given sample videos containing various types of ads that we were tasked with detecting and removing. We also needed to make a simple video player to show the output. I worked with my partner, Jaskaran Khalsa, on this project.

The first step in solving this problem was to divide the video clips into shots. A new shot was defined if it passed a difference threshold between one frame and the next, denoting a significant camera change, or start of an advertisement.

After getting a list of shots (start and end frames), we analysed each one in various categories including audio levels (frequency and amplitude), color, luminance, saturation, blank / black frames and transition type elements at the start and end of the shot. Basically, large differences in shot data and especially the presence of transitions meant a high probability that the shot was an advertisement. Several types of ads, like those with a white background (and often accompanied by loud music) were very easy to detect. The difficult ads typically had close or similar color levels as the previous/following shot and we had to rely on audio / transition data.

We were given 4 sample videos to test/tune our algorithm, but then given blind tests on the final day of the class, needing to remove ads that we had never seen before (also in new videos). Our program performed very well and removed all of the advertisements seamlessly with no false positives. If we had more time, we would extend the ad detection to work using a Machine Learning classifier with the data we gathered from each shot and the sample advertisements, using the data to train the classifier.

Making the video player was fairly straight forward. We used the Java Swing library to create a slider for the video, with buttons to pause and start. The most difficult part was syncing the audio and video thread. To do this, we made the video speed up or slow down in small increments to stay in sync with the audio (which we used as a base reference for real time speed). We figured that distortion in audio would be far more noticeable than very small changes in the frame rate of the video.