Frame extraction hangs when pyav is imported alongside mvextractor

[deepsworld]

Hi @LukasBommes,

Thank you very much for your excellent work. I was able to compile this into a conda package here: https://anaconda.org/necla-ml/mv-extractor which can be installed with conda install -c necla-ml mv-extractor.

However, when ever I use mvextractor alongside pyav (conda install av -c conda-forge). The extractor gets stuck at this line: https://github.com/LukasBommes/mv-extractor/blob/master/src/mvextractor/video_cap.cpp#L199. Can you please let me know if you have any idea about why this could be happening. I am guessing this is caused because pyav also creates a wrapper around the FFMPEG library which could be conflicting with the mvextractor somehow. Here is a small script to reproduce the bug:

import av
# if we remove the av import it works fine 

def extract_motion_vectors(video_pth):
    from mvextractor.videocap import VideoCap

    cap = VideoCap()
    # open the video file
    ret = cap.open(video_pth)

    idx = 0
    while ret:
        # read next video frame and corresponding motion vectors
        ret, frame, motion_vectors, frame_type, timestamp = cap.read()
        if not ret:
            break
        print(frame.shape)
        idx += 1
    
    cap.release()

if __name__ == '__main__':
    outputs = extract_motion_vectors('<path to video file>')
    # import av
    # it works fine if av is imported after the extraction

[LukasBommes]

Hey @deepsworld,

thanks for the great issue! Glad to know that compiling mv-extractor into a conda package works.
Since I have never used pyav before, I can only make some guesses also, which you could try looking into.

1. Maybe, pyav replaces some of the libraries (*.so and *.a files) against which mv-extractor is compiled and which are expected to be in place and unaltered.
2. Before compiling ffmpeg, I apply a patch to some of the source files. This patch is needed for the timestamp feature provided in mv-extractor. As pyav won't have this patch included, this may cause problems.

But I guess, it has got to do with point (1) as you also suggested already.