Skip to content

Commit

Permalink
MatViT README edits
Browse files Browse the repository at this point in the history
PiperOrigin-RevId: 575346989
  • Loading branch information
Scenic Authors committed Oct 23, 2023
1 parent eafec9b commit 9cc66e8
Show file tree
Hide file tree
Showing 2 changed files with 7 additions and 7 deletions.
8 changes: 4 additions & 4 deletions scenic/projects/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -76,10 +76,10 @@
> method to train spatially-aware vision transformer features.
* [MatViT](matvit)
> MatViT ([paper](https://arxiv.org/abs/2310.07707)) is a nested ViT
> architecture designed to offer elasticity in a variety of deployment
> constraints, where each Feed Forward Network (FFN) block of a MatViT
> model is jointly optimized with a few nested smaller FFN blocks.
> MatViT is a MatFormer ([paper](https://arxiv.org/abs/2310.07707)) based
> nested ViT architecture designed to offer elasticity in a variety of
> deployment constraints, where each Feed Forward Network (FFN) block of a
> MatViT model is jointly optimized with a few nested smaller FFN blocks.
* [MBT](mbt)

Expand Down
6 changes: 3 additions & 3 deletions scenic/projects/matvit/README.md
Original file line number Diff line number Diff line change
@@ -1,10 +1,10 @@
### Matryoshka Vision Transformers
### Matryoshka Vision Transformers (MatViT)

This project implements MatViT in Scenic.

This is implemented according to the MatViT part in the MatFormer publication, for details see
This is implemented according to the MatViT adaptation in the MatFormer paper, for details see
Devvrit et al., "MatFormer: Nested Transformer for Elastic Inference",
https://arxiv.org/abs/2310.07707. Further optimizations can be applied to speed up the training.
https://arxiv.org/abs/2310.07707. This implementation provides the basic proof of concept and further optimizations can be applied to speed up the training.

If you are interested in using the code and/or pretrained models, please contact
francischen@google.com and kusupati@google.com.

0 comments on commit 9cc66e8

Please sign in to comment.