./keyframe

keyframe: a simple tool for selecting keyframes from SfM (structure from motion) videos, using naive sharpness, correlation and optical flow criteria. Fast but may be inaccurate as no geometrical estimation of the scene is done in the process.

Dependencies

• python3
• numpy
• cv2 (opencv3 python binding)
• pyqt5

Assuming you have set up your path correctly, I will use pip3 and python3 to refer to those executables in your python3 installation.

Linux (Ubuntu)

• sudo apt-get install python3-pyqt5
• pip3 install numpy
• Compile OpenCV 3 with python3 binding (3.2.0 was used during development). You can follow the steps.

Windows

• Run pip3 install pyqt5
• Install numpy, opencv_python‑3.2.0 prebuilt wheels from here by first downloading the .whl files and then pip3 install <filename>.whl

Usage

1. Run ./run.py or just double-click on run.py if you are using windows.
2. Load the video file
3. Enable or disable filters
   • Filters will be run from top to bottom (first sharpness, then correlation, finally optical flow)
   • The output of a previous filter will be the input of the next filter
   • The order of filters CANNOT be changed (for now)
   • You can have all or no filters enabled
4. Click Run
5. Tune parameters, Preview or Export
6. You can save the tuned filter parameters into a file for later reuse.

Filter Details

Sharpness

• Z-score, we will reject frames whose single-tailed z-score of sharpness value (within the distribution of Window Size frames) is greater than this value. Note that, this score is actually median absolute deviation, used here to compensate for outliers. Here is an NIST page for reference.
• Window Size, how many frames do we consider to be in the same sharpness distribution. In other words, the window size of moving median for the sharpness distribution.

Correlation

• Threshold, between 0 and 1, we will remove the first of two consecutive frames if the correlation of them is higher than threshold (since they appear to be still frames).

Optical Flow

• Threshold, minimum number of pixels for the apparent camera motion. We will try to reject if the camera motion is too small. Sometimes, this goal cannot be achieved (motion between frames too large or lost tracking) and we will accept the frame with value closest to the threshold.

Filter Result Statistics

• Statistics for filter result will show after a filter finishes.
• They will be in the form of [Input # of frames] => [Output # of frames]

tl;dr

• Key Frame Selection for Structure and Motion

License

Released under The MIT License
Copyright 2017 Yifei Zhang