Video Compression and PCA

Intermediate frame interpolation using optical flow with FlowNet2
Explore the repository»

View Problem Statement View Report

tags : video compression, image compression, adaptive bitrate, natural images, pca, ffmpeg, digital video

About The Project

This project deals with the task of calculating the principal components of natural images and video frame compression. In the first task, we extract a large number of patches from natural images of the Berkeley segmentation dataset and visualize the principal components as images. In the second task we try to understand the bit rate below which it is better to downsample the video spatially by 2, compress the video and then upsample rather directly compress at the desired bit rate. A detailed description and analysis of the results are available in the Report.

Built With

This project was built with

• python v3.8
• The environment used for developing this project is available at environment.yml.

Getting Started

Clone the repository into a local machine using

git clone https://github.com/vineeths96/Video-Compression-and-PCA

Prerequisites

Create a new conda environment and install all the libraries by running the following command

conda env create -f environment.yml

The dataset used in this project will be automatically downloaded to the input/ folder. To test on other datasets, download them and put them in the input/ folder.

Instructions to run

Principal components of natural images

To find and plot the principal components of natural images run the following command. Set the parameters for patch extraction in the parameters file. This will generate the images and store it in this folder.

python natural_image_pca.py

Image compression and Resolution

To find the bit rate below which it is better to downsample the video spatially by 2, compress the video and then upsample rather directly compress at the desired bit rate, run the following command. Set the parameters for this experiment in the parameters file. This will compress the video using H.264 with ffmpeg and store the frames and video in this folder.

python image_compression.py

Results

A detailed description of methods and analysis of the results are available in the Report.

We extract 8 × 8 patches from each of these images and perform Principal Component Analysis (PCA) on the patches. We visualize the principal components as images with different number of patches per image. As we would expect, the higher the number of patches, better the results are. We observe that the results closely agrees with the current literature.

100 Patches per Image	1000 Patches per Image
10000 Patches per Image	25000 Patches per Image

We find the bit rate switching for a given video. We measure the quality in terms of mean squared error in the luminance between the original frame and reconstructed frame. Clearly, we can see that there is a switching between the two methods. The switching bit rate is around 400 Kbps — above which compression-decompression in original dimensions is better and below which downsampling - compression - decompression - upsampling is better.

Compress - Decompress	Downsample - Compress - Decompress - Upsample

Comparision

License

Distributed under the MIT License. See LICENSE for more information.

Contact

Vineeth S - vs96codes@gmail.com

Project Link: https://github.com/vineeths96/Video-Compression-and-PCA