-
Notifications
You must be signed in to change notification settings - Fork 440
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
PiperOrigin-RevId: 575346989
- Loading branch information
Scenic Authors
committed
Oct 23, 2023
1 parent
eafec9b
commit 9cc66e8
Showing
2 changed files
with
7 additions
and
7 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,10 +1,10 @@ | ||
### Matryoshka Vision Transformers | ||
### Matryoshka Vision Transformers (MatViT) | ||
|
||
This project implements MatViT in Scenic. | ||
|
||
This is implemented according to the MatViT part in the MatFormer publication, for details see | ||
This is implemented according to the MatViT adaptation in the MatFormer paper, for details see | ||
Devvrit et al., "MatFormer: Nested Transformer for Elastic Inference", | ||
https://arxiv.org/abs/2310.07707. Further optimizations can be applied to speed up the training. | ||
https://arxiv.org/abs/2310.07707. This implementation provides the basic proof of concept and further optimizations can be applied to speed up the training. | ||
|
||
If you are interested in using the code and/or pretrained models, please contact | ||
francischen@google.com and kusupati@google.com. |