Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add pytorch ckpts for crnn & mobilenet_v3_large #487

Merged
merged 1 commit into from
Sep 22, 2021
Merged

Conversation

charlesmindee
Copy link
Collaborator

This PR adds pytorch ckpts for crnn_vgg_16_bn and mobilenet_v3_large, here is the benchmark performed with the ckpts:

FUNSD
Text Detection - Recall: 80.98%, Precision: 85.28%, Mean IoU: 68.91%
Text Recognition - Accuracy: 85.85% (unicase: 86.76%)
OCR - Recall: 67.90% (unicase: 68.52%), Precision: 71.51% (unicase: 72.17%), Mean IoU: 68.91%

CORD
Text Detection - Recall: 80.48%, Precision: 66.62%, Mean IoU: 56.98%
Text Recognition - Accuracy: 92.53% (unicase: 92.92%)
OCR - Recall: 70.80% (unicase: 71.12%), Precision: 58.60% (unicase: 58.86%), Mean IoU: 56.98%

RECEIPTS
Text Detection - Recall: 84.70%, Precision: 85.62%, Mean IoU: 72.19%
Text Recognition - Accuracy: 91.70% (unicase: 92.45%)
OCR - Recall: 76.87% (unicase: 77.51%), Precision: 77.70% (unicase: 78.34%), Mean IoU: 72.19%

IDS
Text Detection - Recall: 66.32%, Precision: 59.12%, Mean IoU: 48.83%
Text Recognition - Accuracy: 64.81% (unicase: 67.50%)
OCR - Recall: 44.58% (unicase: 46.75%), Precision: 39.74% (unicase: 41.67%), Mean IoU: 48.83%

INVOICES
Text Detection - Recall: 69.46%, Precision: 72.85%, Mean IoU: 61.74%
Text Recognition - Accuracy: 90.12% (unicase: 91.59%)
OCR - Recall: 64.03% (unicase: 65.06%), Precision: 67.15% (unicase: 68.23%), Mean IoU: 61.74%

TAX FORM US
Text Detection - Recall: 80.74%, Precision: 92.92%, Mean IoU: 70.90%
Text Recognition - Accuracy: 84.30% (unicase: 84.83%)
OCR - Recall: 78.21% (unicase: 78.58%), Precision: 90.02% (unicase: 90.45%), Mean IoU: 70.90%

@charlesmindee charlesmindee added type: enhancement Improvement module: models Related to doctr.models framework: pytorch Related to PyTorch backend topic: text detection Related to the task of text detection topic: text recognition Related to the task of text recognition labels Sep 21, 2021
@charlesmindee charlesmindee added this to the 0.4.0 milestone Sep 21, 2021
@charlesmindee charlesmindee self-assigned this Sep 21, 2021
Copy link
Contributor

@fg-mindee fg-mindee left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! There are some rather significant differences to be noticed compared to their TF counterparts 🤔

@codecov
Copy link

codecov bot commented Sep 21, 2021

Codecov Report

Merging #487 (69e267f) into main (5d1073f) will not change coverage.
The diff coverage is n/a.

Impacted file tree graph

@@           Coverage Diff           @@
##             main     #487   +/-   ##
=======================================
  Coverage   94.99%   94.99%           
=======================================
  Files         108      108           
  Lines        4177     4177           
=======================================
  Hits         3968     3968           
  Misses        209      209           
Flag Coverage Δ
unittests 94.99% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
...s/detection/differentiable_binarization/pytorch.py 97.43% <ø> (ø)
doctr/models/recognition/crnn/pytorch.py 98.90% <ø> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 5d1073f...69e267f. Read the comment docs.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
framework: pytorch Related to PyTorch backend module: models Related to doctr.models topic: text detection Related to the task of text detection topic: text recognition Related to the task of text recognition type: enhancement Improvement
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants