README.md

SkelMAE

This repository contains the source code for our paper: Skeleton-based Self-Supervised Feature Extraction for Improved Dynamic Hand Gesture Recognition

Updates

• The complete code can be found here

Installation

Create and activate conda environment:

conda create -n skelmae python=3.10
conda activate skelmae

Install all dependencies:

pip install -r requirements.txt

Training

Dowload the evaluation datasets:

• Briareo
• IPN Hand
• SHREC'17.

Hand Pose Extraction

We use MediaPipe framework to extarct hand poses from RGB images (for subsets (training,validation and testing).

• For training set:

python extract_hand_poses.py  --data_dir ./Path/to/IPN_Hand/ 
                              --annotations_file ./Path/to/IPN_train_annotations.txt 
                              --subset training-set
                              --save_dir ./datasets/IPN_Hand/Landmarks/

• For testing set

python extract_hand_poses.py  --data_dir ./Path/to/IPN_Hand/ 
                              --annotations_file ./Path/to/IPN_test_annotations.txt 
                              --subset test-set
                              --save_dir ./datasets/IPN_Hand/Landmarks/

Training

bash train.sh --config_file configs/ipn_hand_config.yaml

Evaluation

bash eval.sh --config_file configs/ipn_hand_config.yaml

Citation

If you find this repo useful, please consider citing our paper

ref

We thank MAE and STGCN for making their code available