Augmenting Knee Rehabilitation Replays Using MediaPipe Pose Estimation

Overview

This project implements a methodology for analysing knee angles from video footage using MediaPipe Pose estimation. It provides near real-time visualisation $${\large\color{dodgerblue}ⓘ }$$ and analysis of knee movements, supporting both front-view and side-view perspectives.

Note

Near real-time visualisation of MP4 videos depends on the computer specification.
Hint: If using a decade old laptop for fast scrolling and A-B sequence analysis, consider capturing first a screen cast of the generated augmented video replay.

As a stand-alone application, this source code is the second part of the three stage processing workflow, which is also a part of it's parent project codebase ¹ intended for home use and advancements of near-future analytical healthcare systems. See more: "Towards nation-wide analytical healthcare infrastructures: A privacy-preserving augmented knee rehabilitation case study" ¹².

Features

• Near real-time knee angle detection and measurement
• Support for both left and right knee analysis
• Front-view and side-view analysis capabilities
• CSV export of tracking data
• Real-time visualization with angle plots
• Incorrect knee movement detection

Requirements

• Python 3.8+
• OpenCV
• MediaPipe
• Matplotlib
• NumPy

Installation

If running the project on a computer with Python and OpenCV installed, install Google MediaPipe by:

pip install mediapipe

or by preparing your own requirements.txt for project deployments on multiple machines of similar specifications:

pip install -r requirements.txt

Usage

To run this application, at Windows Command Prompt, or in MacOS/Linux Terminal, use the following sytax:

python main.py video_file output_csv [--export_knee {left,right,both}] [--direction {left,right,forward}]

Arguments
video_file: Path to input video file
output_csv: Path for output CSV file
--export_knee: Specify which knee to analyze (default: both)
--direction: Override foot direction detection

Citation

If using our code, algorithms or models for your research, please cite ¹ or use BibTeX format:

@inproceedings{bbacic2024simple,
    author={Bačić, Boris and Vasile, Claudiu and Feng, Chengwei and Ciucă, Marian},
    title={Towards nation-wide analytical healthcare infrastructures: A privacy-preserving augmented knee rehabilitation case study},
    booktitle = {Conference on Innovative Technologies in Intelligent Systems & Industrial Applications (CITISIA 2024)},
    year = {2024}
    pages={10},
    date = {13-15 Dec.},
    address = {Sydney, NSW},
}

If using LaTeX or Overleaf, to preserve Unicode/special characters, a recommended BibTeX format is:

@inproceedings{bbacic2024simpleTeX,
    author={Ba{\vc}i{\'c}, B and Vasile, C and Feng, C and Ciuc{\ua}, M},
    title={Towards nation-wide analytical healthcare infrastructures: A privacy-preserving augmented knee rehabilitation case study},
    booktitle = {Conference on Innovative Technologies in Intelligent Systems & Industrial Applications (CITISIA 2024)},
    year = {2024},
    pages={10},
    date = {13-15 Dec.},
    address = {Sydney, NSW},
}

 

Contributors

Contributor	Description
Claudiu Vasile @claudiunz	Python code, incremental prototyping, testing, and development. Github main project tnwahi-appakrcs.
Dr Marian G. Ciucă	Computational geometry and trigonometric equations for knee angle calculations.
Chengwei Feng	Dataset labelling verification, models and code templates alternatives.
Dr Boris Bačić @bbacic	Project leader, supervision, codebase development, testing, reviewing, and integration1. GitHub project reviewer and co-contributor.1)

¹⁾Note: https://github.com/bbacic/tnwahi-appakrcs/ is a contributing fork of tnwahi-appakrcs.

Version History of the tnwahi-appakrcs Project

Version	Date	Summary/Action/Rationale/Acknowledgements	Project/Filename
3.2	Dec 2024	CITISIA 2024 publication 1. Refinements, issues and bug fixes of PoC (ver. 3) for the scope of the publication.	main.py
3	Nov 2024	Front and side camera view with extended diagnostic information processing (from ver. 2 PoC).	main.py <- script.py
2	May 2024	Privacy-preserving augmented video analysis with diagnostic information streaming and visualisation (PoC).	script.py
1	Aug 2023	General project prototyping and investigations. Beta version of proof-of-concept (PoC).	script.py

 

Reference

Footnotes

1. Bačić, B., Claudiu Vasile, Feng, C., & Ciucă, M. G. (2024, 13-15 Dec.). Towards nation-wide analytical healthcare infrastructures: A privacy-preserving augmented knee rehabilitation case study. Presented at the meeting of the Conference on Innovative Technologies in Intelligent Systems & Industrial Applications (CITISIA 2024), Sydney, NSW. [In print]. ↩ ↩² ↩³ ↩⁴ ↩⁵

2. Bačić, B., Claudiu Vasile, Feng, C., & Ciucă, M. G. (2024, 13-15 Dec.). Towards nation-wide analytical healthcare infrastructures: A privacy-preserving augmented knee rehabilitation case study. Presented at the meeting of the Conference on Innovative Technologies in Intelligent Systems & Industrial Applications (CITISIA 2024), Sydney, NSW. [ArXiv.org preprint]. Accessed 31 Dec. 2024 at: https://arxiv.org/abs/2412.20733. ↩