Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

add ViT as classification model TF and PT #1050

Merged
merged 11 commits into from
Sep 9, 2022
Merged

add ViT as classification model TF and PT #1050

merged 11 commits into from
Sep 9, 2022

Conversation

felixdittrich92
Copy link
Contributor

@felixdittrich92 felixdittrich92 commented Sep 7, 2022 •
edited
Loading

This PR:

  • adds ViT as classification model, which can be used as backbone for ViTSTR/ParSeq

Any feedback or ideas for improvements are very welcome 🤗

validated via test runs both models trains well

@codecov
Copy link

codecov bot commented Sep 7, 2022 •
edited
Loading

Codecov Report

Merging #1050 (806494f) into main (1cc073d) will increase coverage by 0.09%.
The diff coverage is 98.95%.

@@            Coverage Diff             @@
##             main    #1050      +/-   ##
==========================================
+ Coverage   94.94%   95.03%   +0.09%     
==========================================
  Files         135      141       +6     
  Lines        5634     5818     +184     
==========================================
+ Hits         5349     5529     +180     
- Misses        285      289       +4
Flag Coverage Δ
unittests 95.03% <98.95%> (+0.09%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
doctr/models/classification/zoo.py 100.00% <ø> (ø)
doctr/models/classification/vit/pytorch.py 97.50% <97.50%> (ø)
doctr/models/classification/vit/tensorflow.py 97.82% <97.82%> (ø)
doctr/models/classification/__init__.py 100.00% <100.00%> (ø)
doctr/models/classification/vit/__init__.py 100.00% <100.00%> (ø)
doctr/models/modules/__init__.py 100.00% <100.00%> (ø)
doctr/models/modules/transformer/pytorch.py 100.00% <100.00%> (ø)
doctr/models/modules/transformer/tensorflow.py 98.98% <100.00%> (+0.23%) ⬆️
...octr/models/modules/vision_transformer/__init__.py 100.00% <100.00%> (ø)
doctr/models/modules/vision_transformer/pytorch.py 100.00% <100.00%> (ø)
... and 2 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

@felixdittrich92 felixdittrich92 mentioned this pull request Sep 8, 2022
Closed
2 tasks
@felixdittrich92 felixdittrich92 changed the title [DRAFT] Option 2 ViT [DRAFT] add ViT as classification model TF and PT Sep 8, 2022
@felixdittrich92 felixdittrich92 self-assigned this Sep 8, 2022
@felixdittrich92 felixdittrich92 added module: models Related to doctr.models framework: pytorch Related to PyTorch backend framework: tensorflow Related to TensorFlow backend topic: character classification Related to the task of character classification type: new feature New feature labels Sep 8, 2022
@felixdittrich92 felixdittrich92 linked an issue Sep 8, 2022 that may be closed by this pull request
Release tracker - v0.6.0 #791
Closed
85 tasks
@felixdittrich92 felixdittrich92 removed a link to an issue Sep 8, 2022
Release tracker - v0.6.0 #791
Closed
85 tasks
@felixdittrich92 felixdittrich92 added this to the 0.6.0 milestone Sep 8, 2022
@felixdittrich92 felixdittrich92 changed the title [DRAFT] add ViT as classification model TF and PT add ViT as classification model TF and PT Sep 8, 2022
@felixdittrich92 felixdittrich92 marked this pull request as ready for review September 8, 2022 12:01
odulcy-mindee
odulcy-mindee approved these changes Sep 9, 2022
View reviewed changes
Copy link
Collaborator

@odulcy-mindee odulcy-mindee left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Really cool, thanks @felixdittrich92 ! 👏 🚀

@felixdittrich92
Copy link
Contributor Author

Really cool, thanks @felixdittrich92 ! clap rocket

yeah now we can continue with ViTSTR and ParSeq 🤗

@felixdittrich92 felixdittrich92 merged commit 05f5777 into mindee:main Sep 9, 2022
@felixdittrich92 felixdittrich92 deleted the vit-classification branch September 9, 2022 12:26
@felixdittrich92
Copy link
Contributor Author

@frgfm late review maybe ? 😅

frgfm
frgfm reviewed Sep 12, 2022
View reviewed changes
Copy link
Collaborator

@frgfm frgfm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks Felix! Looks good, I added a few comments :)

docs/source/modules/models.rst Show resolved Hide resolved
doctr/models/classification/vit/pytorch.py Show resolved Hide resolved
doctr/models/classification/vit/pytorch.py Show resolved Hide resolved
doctr/models/modules/transformer/pytorch.py Show resolved Hide resolved
@frgfm frgfm mentioned this pull request Sep 16, 2022
Merged
@felixdittrich92 felixdittrich92 mentioned this pull request Sep 26, 2022
Closed
85 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@frgfm frgfm frgfm left review comments

@odulcy-mindee odulcy-mindee odulcy-mindee approved these changes

@charlesmindee charlesmindee Awaiting requested review from charlesmindee

@aminemindee aminemindee Awaiting requested review from aminemindee

Assignees

@felixdittrich92 felixdittrich92

Labels
framework: pytorch Related to PyTorch backend framework: tensorflow Related to TensorFlow backend module: models Related to doctr.models topic: character classification Related to the task of character classification type: new feature New feature
Projects
None yet
Milestone
0.6.0
Development

Successfully merging this pull request may close these issues.
3 participants
@felixdittrich92 @frgfm @odulcy-mindee