Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[TDL-22316] : Handle ChunkEncodingError & JSONDecodeError #36

Merged
merged 2 commits into from
Mar 23, 2023

Conversation

shantanu73
Copy link
Contributor

@shantanu73 shantanu73 commented Mar 21, 2023

Description of change

After referring to customer logs :

2023-03-20 03:50:33,745Z    tap - INFO HTTP request to "contact_custom_field_values" endpoint took 2.339s, returned status code 200
2023-03-20 03:50:33,746Z    tap - ERROR ("Connection broken: ConnectionResetError(104, 'Connection reset by peer')", ConnectionResetError(104, 'Connection reset by peer'))
2023-03-20 03:50:33,746Z    tap - ERROR response content: b''
2023-03-20 03:50:33,747Z    tap - CRITICAL 'ChunkedEncodingError' object is not subscriptable
2023-03-20 03:50:33,747Z    tap - Traceback (most recent call last):
2023-03-20 03:50:33,747Z    tap -   File "/code/orchestrator/tap-env/lib/python3.9/site-packages/urllib3/response.py", line 436, in _error_catcher
2023-03-20 03:50:33,747Z    tap -     yield
2023-03-20 03:50:33,747Z    tap -   File "/code/orchestrator/tap-env/lib/python3.9/site-packages/urllib3/response.py", line 763, in read_chunked
2023-03-20 03:50:33,747Z    tap -     self._update_chunk_length()
2023-03-20 03:50:33,747Z    tap -   File "/code/orchestrator/tap-env/lib/python3.9/site-packages/urllib3/response.py", line 693, in _update_chunk_length
2023-03-20 03:50:33,747Z    tap -     line = self._fp.fp.readline()
2023-03-20 03:50:33,747Z    tap -   File "/root/.pyenv/versions/3.9.6/lib/python3.9/socket.py", line 704, in readinto
2023-03-20 03:50:33,748Z    tap -     return self._sock.recv_into(b)
2023-03-20 03:50:33,748Z    tap -   File "/root/.pyenv/versions/3.9.6/lib/python3.9/ssl.py", line 1241, in recv_into
2023-03-20 03:50:33,748Z    tap -     return self.read(nbytes, buffer)
2023-03-20 03:50:33,748Z    tap -   File "/root/.pyenv/versions/3.9.6/lib/python3.9/ssl.py", line 1099, in read
2023-03-20 03:50:33,748Z    tap -     return self._sslobj.read(len, buffer)
2023-03-20 03:50:33,748Z    tap - ConnectionResetError: [Errno 104] Connection reset by peer

It's observed that when an API from Active-Campaign returns
an empty byte b'' response for a 200 status code, then, ChunkedEncodingError/JSONDecodeError is thrown in tap's response.

  1. Added logic to handle ChunkedEncodingError and JSONDecodeError.
  2. Added necessary unit test case.

Manual QA steps

  • Referred to customer logs and wrote a test API to test the errors.

Risks

  • Low Risk

Rollback steps

  • revert this branch

1) Added logic to handle ChunkedEncodingError and JSONDecodeError.
2) Added necessary unit test case.
1) Improved exception throwing and handling logic.
2) Added backoff for ConnectionResetError.
3) Added unit test case for ConnectionResetError.
@shantanu73 shantanu73 force-pushed the TDL-22316-chunkencoding-error branch from fcb7dcb to 4bf4a09 Compare March 23, 2023 10:49
@shantanu73 shantanu73 merged commit 4f85b5f into master Mar 23, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants