Segment Anything Model (SAM) has attracted considerable attention from the community since its inception. However, the formidable model size and demanding computational requirements of SAM have rendered it cumbersome for deployment on resource-constrained devices. To mitigate these constraints, many efforts have been made to effectively make SAM more efficient and lightweight. This repository provides a summary of those efficient segment anything models.
If you find this repository helpful, please consider Stars ⭐ or Sharing ⬆️. Thanks.
- 🚀 August 7, 2024: Updated new related works before 07/08/2024 in this GitHub.
- 🚀 March 22, 2024: Updated new related works before 20/03/2024 in this GitHub.
Fast Segment Anything
Xu Zhao, Wenchao Ding, Yongqi An, Yinglong Du, Tao Yu, Min Li, Ming Tang, Jinqiao Wang
Institute of Automation, Chinese Academy of Sciences, Beijing, China
Paper: [Arxiv] Code:
Description: The Fast Segment Anything Model(FastSAM) is a CNN Segment Anything Model trained using only 2% of the SA-1B dataset published by SAM authors.
Lite-SAM Is Actually What You Need for Segment Everything
Jianhai Fu, Yuanjie Yu, Ningchuan Li, Yi Zhang, Qichao Chen, Jianping Xiong, Jun Yin, and Zhiyu Xiang
Zhejiang Dahua Technology, Zhejiang University, Hangzhou, China
Paper: [Arxiv]
Description: Lite-SAM is an efficient end-to-end solution for the SegEvery task designed to reduce computational costs and redundancy.
FASTER SEGMENT ANYTHING: TOWARDS LIGHTWEIGHT SAM FOR MOBILE APPLICATIONS
Chaoning Zhang, Dongshen Han, Sheng Zheng, Jinwoo Choi, Tae-Ho Kim, Choong Seon Hong
Kyung Hee University
Paper: [Arxiv] Code:
Description: MobileSAM is a lightweight SAM suitable for resource-constrained mobile devices.
EdgeSAM: Prompt-In-the-Loop Distillation for On-Device Deployment of SAM
Chong Zhou, Xiangtai Li, Chen Change Loy, Bo Dai
S-Lab, Nanyang Technological University, Shanghai Artificial Intelligence Laborator
Paper: [Arxiv] Code:
Description: EdgeSAM is an accelerated variant of the Segment Anything Model (SAM), optimized for efficient execution on edge devices with minimal compromise in performance.
EfficientSAM: Leveraged Masked Image Pretraining for Efficient Segment Anything
Yunyang Xiong et al.
Meta AI Research
Paper: [Arxiv] Code:
Description: EfficientSAMs is lightweight SAM model that exhibits decent performance with largely reduced complexity. The idea is based on leveraging masked image pretraining, SAMI, which learns to reconstruct features from SAM image encoder for effective visual representation learning.
EfficientViT-SAM: Accelerated Segment Anything Model Without Performance Loss
Zhuoyang Zhang, Han Cai, Song Han
Tsinghua University, MIT, NVIDIA
Paper: [Arxiv] Code:
Description: EfficientViT-SAM is a new family of accelerated segment anything models. It retain SAM’s lightweight prompt encoder and mask decoder while replacing the heavy image encoder with EfficientViT
RepViT-SAM: Towards Real-Time Segmenting Anythings
Ao Wang, Hui Chen, Zijia Lin, Jungong Han, Guiguang Ding
Tsinghua University, The University of Sheffield
Paper: [Arxiv] Code:
Description: RepViT achieves the state-of-the-art performance and latency trade-off on mobile devices by incorporating efficient architectural designs of ViTs into CNNs. Here, to achieve real-time segmenting anything on mobile devices, following [27], this work replace the heavyweight image encoder in SAM with RepViT model, ending up with the RepViT-SAM model.
SAM-LIGHTENING: A LIGHTWEIGHT SEGMENT ANYTHING MODEL WITHIN DILATED FLASH ATTENTION TO ACHIEVE 30× ACCELERATION
Yanfei Song, Bangzheng Pu, Peng Wang, Dong Dong, Hongxu Jiang, Yiqing Shen
Beihang University, Beihang Hangzhou Innovation Institute, Johns Hopkins University
Paper: [Arxiv] Code: [Anonymous Github]
Description: SAM-Lightening revolutionizes image segmentation by introducing a highly efficient, lightweight model. This model features an innovative Dilated Flash Attention mechanism, enabling rapid inference and minimal memory usage, ideal for applications in real-time environments and resource-constrained devices.
TinySAM: Pushing the Envelope for Efficient Segment Anything Model
Han Shu, Wenshuo Li, Yehui Tang, Yiman Zhang, Yihao Chen, Houqiang Li, Yunhe Wang, Xinghao Che
University of Science and Technology of China, Huawei Noah’s Ark Lab
Paper: [Arxiv] Code:
Description: TinySAM is a framework can obtain a tiny segment anything model while maintaining the strong zero-shot performance.
SlimSAM: 0.1% Data Makes Segment Anything Slim
Zigeng Chen, Gongfan Fang, Xinyin Ma, Xinchao Wang
Learning and Vision Lab, National University of Singapore
Paper: [Arxiv] Code:
Description: SlimSAM is a novel data-efficient SAM compression method that achieves superior performance with extremely less training data. The essence of SlimSAM is encapsulated in the alternate slimming framework which effectively enhances knowledge inheritance under severely limited training data availability and exceptional pruning ratio.
Expediting Large-Scale Vision Transformer for Dense Prediction without Fine-tuning
Weicong Liang et al.
Key Laboratory of Machine Perception, Peking University, ETH Zurich
Paper: [Arxiv] Code:
Description: Expedit-SAM is a method can speed up SAM without any training. This is achieved by a novel token clustering algorithm.
