Adwatch Engine

Adwatch aimed to create a comprehensive workflow for automatic audio and video content detection, particularly focusing on advertisements and promos within TV input streams. The goal was to ensure uninterrupted monitoring of media news campaigns.

Executive Summary

Pixelette Technologies developed the Adwatch Engine for automated ad detection and removal from TV input streams. Utilizing a Linux-Python stack and employing Audfprint, ffmpeg, and moviepy for processing, along with a fusion of VGG-16 and Audfprint for accurate ad detection, the project successfully achieved efficient ad removal. This system serves as a robust tool for monitoring media campaigns by eliminating interruptions caused by commercials, thereby ensuring a streamlined workflow for content analysis.

Project Objective

  • Develop an efficient ad detection system using a dataset of image and audio files.
  • Implement a workflow for processing video files, detecting and removing ads, and outputting ad-free video streams.
  • Integrate the developed solution into a system for end-to-end automated ad removal.

Challenges

  • Accurate conversion of video files to audio for processing.
  • Efficient ad detection by matching fingerprints and utilizing CNN for additional frame detection.
  • Seamless removal of ads from the original input without affecting the video quality.

Solutions

  • Utilized ffmpeg and moviepy for accurate video-to-audio conversion and processing.
  • Employed a fusion of Audfprint and a fine-tuned VGG-16 model for precise ad detection.
  • Implemented an efficient ad removal process ensuring the output video remains intact and of high quality.

Project Timeline

  • Phase 1: Dataset Generation and Pre-processing
  • Phase 2: Training and Fine-tuning VGG-16 Model
  • Phase 3: Audfprint Database Generation
  • Phase 4: Fusion of Audfprint and VGG-16 for Ad Detection
  • Phase 5: Testing and Evaluation

Team Composition:

  • Project Manager
  • Machine Learning Engineers
  • Backend Developers
  • Data Annotation Team
  • Quality Assurance Engineers

Risk Management:

Identified risks like inaccurate ad detection and data misclassification. Implemented robust training and evaluation processes to mitigate these risks.

Quantifiable Results

  • Achieved over 95% accuracy in ad detection and removal.
  • Significantly improved the efficiency of monitoring media campaigns by reducing the time spent on manual ad filtering.

Lessons Learned

Accurate dataset generation and effective fusion of audio and visual processing techniques are crucial for successful ad detection and removal.

Client Feedback

Adwatch appreciated the accuracy and efficiency of the Adwatch Engine in automating ad detection and removal, which significantly streamlined their media monitoring processes.

Conclusion

The Adwatch Engine developed by Pixelette Technologies successfully addressed Adwatch’s need for an automated ad detection and removal system. The project demonstrated the effectiveness of employing a combination of audio fingerprinting and visual processing in automating content filtering, significantly improving the efficiency of monitoring media campaigns.