Skip to content

Official Tensorflow implementation of ConvNeXt-ChARM: ConvNeXt-based Transform for Efficient Neural Image Compression.

License

Notifications You must be signed in to change notification settings

ahmedgh970/convnext-charm

Repository files navigation

ConvNeXt-ChARM: ConvNeXt-based Transform for Efficient Neural Image Compression

Official TensorFlow implementation of ConvNeXt-ChARM: ConvNeXt-based Transform for Efficient Neural Image Compression.

Tags

Swin Transformer ConvNeXt Learning-based Codecs Image Compression TensorFlow

Overall ConvNeXt-ChARM Framework

ConvNeXt-ChARM framework

Disclaimer

Please do not hesitate to open an issue to inform of any problem you may find within this repository. Also, you can email me for questions or comments.

Documentation

Requirements

Python >= 3.6 tensorflow_compression tensorflow_datasets tensorflow_addons einops

All packages used in this repository are listed in requirements.txt. To install those, run:

pip install -r requirements.txt

Folder Structure

convnext-charm
│
├── conv-charm.py                 # Conv-ChARM Model
├── conv-charm_lrp.py             # Conv-ChARM Model with latent residual prediction (LRP) 
├── convnext-charm.py             # ConvNeXt-ChARM Model
├── convnext-charm_lrp.py         # ConvNeXt-ChARM Model with latent residual prediction (LRP) 
├── swint-charm.py                # SwinT-ChARM Model
├── swint-charm_lrp.py            # SwinT-ChARM Model with latent residual prediction (LRP) 
├── utils.py                      # Utility scripts
│
├── layers/
│   └── convNext.py/              # ConvNeXt block layers
│   └── swinTransformer.py/       # Swin Transformer block layers
|
└── figures/                      # Documentation figures

CLI Usage

Every model can be trained and tested individually using:

python convnext-charm.py train
python convnext-charm.py evaluate

Rate-Distortion coding performance

Table 1. BD-rate↓ performance of BPG (4:4:4), SwinT-ChARM, and ConvNeXt-ChARM compared to the VTM-18.0 for the four considered datasets.

Dataset BPG444 SwinT-ChARM ConvNeXt-ChARM
Kodak 20.73% -3.47% -4.90%
Tecnick 27.03% -6.52% -7.56%
JPEG-AI 28.14% -0.23% -1.17%
CLIC21 26.54% -5.86% -7.36%
Average 25.61% -4.02% -5.24%

Citation

If you use this library for research purposes, please cite:

@inproceedings{ghorbel2023convnextcharm,
  title={ConvNeXt-ChARM: ConvNeXt-based Transform for Efficient Neural Image Compression},
  author={Ghorbel, Ahmed and Hamidouche, Wassim and Luce, Morin},
  booktitle={},
  year={2023}
}

License

This project is licensed under the MIT License. See LICENSE for more details

About

Official Tensorflow implementation of ConvNeXt-ChARM: ConvNeXt-based Transform for Efficient Neural Image Compression.

Topics

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages