Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implemented a workaround to deal with the problem that padding with the minimum value causes the output error of MaxPool2D to be maximized only when quantizing with INT8 quantization. #444 #446

Merged
merged 1 commit into from
Aug 9, 2023

Conversation

PINTO0309
Copy link
Owner

1. Content and background

  • MaxPool
    • Implemented a workaround to deal with the problem that padding with the minimum value causes the output error of MaxPool2D to be maximized only when quantizing with INT8 quantization. [OSNet] int8 tflite model - catastrophic accuracy degradation  #444

      image

      Float32 model outputs flattened shape: (512,)
      Int8 model outputs flattened shape: (512,)
      Euclidean Distance: 2.0942165851593018
      

2. Summary of corrections

3. Before/After (If there is an operating log that can be used as a reference)

4. Issue number (only if there is a related issue)

…he minimum value causes the output error of `MaxPool2D` to be maximized only when quantizing with INT8 quantization. #444
@PINTO0309 PINTO0309 merged commit 1c75970 into main Aug 9, 2023
@PINTO0309 PINTO0309 deleted the fix_maxpool_int8 branch August 9, 2023 02:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant