Piano Video is designed to assist in the creation and analysis of data from videos featuring people playing the piano. Video footage is expected to be from an overhead perspective without significant rotation.
-
Import the
piano_video.pymodule:from piano_video import PianoVideo
-
Initialize the module with the path to the video file. Ensure that the video file name is unique as it will be used for storing intermediate files:
pv = PianoVideo("example.mp4")
The following data points can be extracted using the initialized PianoVideo object:
-
pv.transcribed_midi: MIDI data generated during processing, stored as an interval tree. -
pv.hand_landmarks: List of hand landmarks in the format[frame_id, left_hand, right_hand], generated by the Mediapipe Hands library. -
pv.background: The estimated image of the piano without hands. -
pv.keys: Position of the keyboard and each detected key with its corresponding MIDI ID. -
pv.fingers: An adjusted MIDI representation with estimated fingerings. -
pv.sections: Location of frames containing a keyboard as an interval tree.
Ensure that FFmpeg is available in your system's PATH.
Install the required Python packages by running:
pip install -r requirements.txt