Skip to content

http.client.ResponseNotReady raised in 0.11.4 #205

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
tongyifan opened this issue May 18, 2023 · 4 comments
Closed

http.client.ResponseNotReady raised in 0.11.4 #205

tongyifan opened this issue May 18, 2023 · 4 comments

Comments

@tongyifan
Copy link

latest pyodps installed via pip install pyodps --index-url="https://pypi.org/simple"

pyodps: 0.11.4
urllib3: 2.0.2

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/odps/lib/requests/adapters.py", line 551, in send_with_fileio
    r = low_conn.getresponse(buffering=True)
TypeError: getresponse() got an unexpected keyword argument 'buffering'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "main.py", line 195, in write_odps
    DataFrameODPS(data).persist(table_name, partition=partition_keys, drop_table=True, odps=o,
  File "/usr/local/lib/python3.8/dist-packages/odps/df/expr/expressions.py", line 43, in __call__
    return self._func(*args, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/odps/df/expr/expressions.py", line 266, in persist
    return self._handle_delay_call('persist', self, name, partitions=partitions, partition=partition,
  File "/usr/local/lib/python3.8/dist-packages/odps/df/expr/expressions.py", line 43, in __call__
    return self._func(*args, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/odps/df/expr/expressions.py", line 158, in _handle_delay_call
    result = getattr(engine, method)(*args, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/odps/df/backends/core.py", line 740, in persist
    return self._action(*exprs_args_kwargs, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/odps/df/backends/core.py", line 554, in _action
    res = self._execute_dag(dag, ui=ui, async_=async_, n_parallel=n_parallel,
  File "/usr/local/lib/python3.8/dist-packages/odps/df/backends/core.py", line 798, in _execute_dag
    return dag.execute(ui=ui, async_=async_, n_parallel=n_parallel, timeout=timeout,
  File "/usr/local/lib/python3.8/dist-packages/odps/df/backends/core.py", line 337, in execute
    results = self._run(ui, progress_proportion)
  File "/usr/local/lib/python3.8/dist-packages/odps/df/backends/core.py", line 212, in _run
    res = call(ui=ui, progress_proportion=progress_proportion / len(calls))
  File "/usr/local/lib/python3.8/dist-packages/odps/df/backends/core.py", line 191, in __call__
    res = self.run(ui=ui, progress_proportion=progress_proportion)
  File "/usr/local/lib/python3.8/dist-packages/odps/df/backends/core.py", line 690, in run
    result = engine._do_persist(expr_dag, expr, name, **kw)
  File "/usr/local/lib/python3.8/dist-packages/odps/df/backends/pd/engine.py", line 61, in wrapped
    return fun(self, *args, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/odps/df/backends/pd/engine.py", line 359, in _do_persist
    write_table(df, t, ui=ui, cast=cast, overwrite=overwrite, partitions=partitions, partition=partition,
  File "/usr/local/lib/python3.8/dist-packages/odps/df/backends/utils.py", line 251, in write_table
    _write_table_no_partitions(frame, table, ui, cast=cast, overwrite=overwrite, partition=partition,
  File "/usr/local/lib/python3.8/dist-packages/odps/df/backends/utils.py", line 209, in _write_table_no_partitions
    writer.write(gen())
  File "/usr/local/lib/python3.8/dist-packages/odps/models/table.py", line 360, in __exit__
    self.close()
  File "/usr/local/lib/python3.8/dist-packages/odps/models/table.py", line 917, in close
    super(_TableWriter, self).close()
  File "/usr/local/lib/python3.8/dist-packages/odps/models/table.py", line 339, in close
    block_info.writer.close()
  File "/usr/local/lib/python3.8/dist-packages/odps/tunnel/io/writer.py", line 360, in close
    self._req_io.close()
  File "/usr/local/lib/python3.8/dist-packages/odps/rest.py", line 110, in close
    self.__exit__(None, None, None)
  File "/usr/local/lib/python3.8/dist-packages/odps/rest.py", line 118, in __exit__
    self._ctx.__exit__(*exc_info)
  File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__
    next(self.gen)
  File "/usr/local/lib/python3.8/dist-packages/odps/rest.py", line 233, in _request
    yield writer
  File "/usr/local/lib/python3.8/dist-packages/odps/lib/requests/sessions.py", line 352, in __exit__
    self._ctx.__exit__(*exc_info)
  File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__
    next(self.gen)
  File "/usr/local/lib/python3.8/dist-packages/odps/lib/requests/sessions.py", line 723, in _send
    yield writer
  File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__
    next(self.gen)
  File "/usr/local/lib/python3.8/dist-packages/odps/lib/requests/adapters.py", line 554, in send_with_fileio
    r = low_conn.getresponse()
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connection.py", line 440, in getresponse
    raise ResponseNotReady()
http.client.ResponseNotReady
@wjsi
Copy link
Collaborator

wjsi commented May 18, 2023 •
edited
Loading

Could you please provide minimal code to reproduce this error? As code below cannot simply reproduce.

from odps import DataFrame, options
import numpy as np, pandas as pd
pddf = pd.DataFrame(np.random.rand(10000, 10), columns=list("ABCDEFGHIJ"))
df = DataFrame(pddf).persist("test_pd_persist_table", lifecycle=1)

@tongyifan
Copy link
Author

Could you please provide minimal code to reproduce this error? As code below cannot simply reproduce.

from odps import DataFrame, options
import numpy as np, pandas as pd
pddf = pd.DataFrame(np.random.rand(10000, 10), columns=list("ABCDEFGHIJ"))
df = DataFrame(pddf).persist("test_pd_persist_table", lifecycle=1)

But this code will throw ResponseNotReady in my environment.

here's another script to repro this issue

  • requirements.txt
pyodps==0.11.4
pandas
  • main.py
import pandas as pd
from odps import ODPS, DataFrame

if __name__ == "__main__":
    o = ODPS(
        "my_access_id",
        "my_secret_key",
        "my_project",
        "my_endpoint",
    )

    df = pd.DataFrame(
        {
            "id_code": ["1", "2", "3"],
            "ds": pd.to_datetime(["2023-01-01", "2023-01-01", "2023-01-01"]),
            "value": [4, 5, 6],
        }
    )

    DataFrame(df).persist(
        "table_name",
        drop_table=True,
        odps=o,
        lifecycle=1,
    )
  • output
/Users/tongyifan/PycharmProjects/pyodps_debug/venv/bin/python /Users/tongyifan/PycharmProjects/pyodps_debug/main.py 
Traceback (most recent call last):
  File "/Users/tongyifan/PycharmProjects/pyodps_debug/venv/lib/python3.8/site-packages/odps/lib/requests/adapters.py", line 551, in send_with_fileio
    r = low_conn.getresponse(buffering=True)
TypeError: getresponse() got an unexpected keyword argument 'buffering'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/tongyifan/PycharmProjects/pyodps_debug/main.py", line 20, in <module>
    DataFrame(df).persist(
  File "/Users/tongyifan/PycharmProjects/pyodps_debug/venv/lib/python3.8/site-packages/odps/df/expr/expressions.py", line 43, in __call__
    return self._func(*args, **kwargs)
  File "/Users/tongyifan/PycharmProjects/pyodps_debug/venv/lib/python3.8/site-packages/odps/df/expr/expressions.py", line 266, in persist
    return self._handle_delay_call('persist', self, name, partitions=partitions, partition=partition,
  File "/Users/tongyifan/PycharmProjects/pyodps_debug/venv/lib/python3.8/site-packages/odps/df/expr/expressions.py", line 43, in __call__
    return self._func(*args, **kwargs)
  File "/Users/tongyifan/PycharmProjects/pyodps_debug/venv/lib/python3.8/site-packages/odps/df/expr/expressions.py", line 158, in _handle_delay_call
    result = getattr(engine, method)(*args, **kwargs)
  File "/Users/tongyifan/PycharmProjects/pyodps_debug/venv/lib/python3.8/site-packages/odps/df/backends/core.py", line 740, in persist
    return self._action(*exprs_args_kwargs, **kwargs)
  File "/Users/tongyifan/PycharmProjects/pyodps_debug/venv/lib/python3.8/site-packages/odps/df/backends/core.py", line 554, in _action
    res = self._execute_dag(dag, ui=ui, async_=async_, n_parallel=n_parallel,
  File "/Users/tongyifan/PycharmProjects/pyodps_debug/venv/lib/python3.8/site-packages/odps/df/backends/core.py", line 798, in _execute_dag
    return dag.execute(ui=ui, async_=async_, n_parallel=n_parallel, timeout=timeout,
  File "/Users/tongyifan/PycharmProjects/pyodps_debug/venv/lib/python3.8/site-packages/odps/df/backends/core.py", line 337, in execute
    results = self._run(ui, progress_proportion)
  File "/Users/tongyifan/PycharmProjects/pyodps_debug/venv/lib/python3.8/site-packages/odps/df/backends/core.py", line 212, in _run
    res = call(ui=ui, progress_proportion=progress_proportion / len(calls))
  File "/Users/tongyifan/PycharmProjects/pyodps_debug/venv/lib/python3.8/site-packages/odps/df/backends/core.py", line 191, in __call__
    res = self.run(ui=ui, progress_proportion=progress_proportion)
  File "/Users/tongyifan/PycharmProjects/pyodps_debug/venv/lib/python3.8/site-packages/odps/df/backends/core.py", line 690, in run
    result = engine._do_persist(expr_dag, expr, name, **kw)
  File "/Users/tongyifan/PycharmProjects/pyodps_debug/venv/lib/python3.8/site-packages/odps/df/backends/pd/engine.py", line 61, in wrapped
    return fun(self, *args, **kwargs)
  File "/Users/tongyifan/PycharmProjects/pyodps_debug/venv/lib/python3.8/site-packages/odps/df/backends/pd/engine.py", line 359, in _do_persist
    write_table(df, t, ui=ui, cast=cast, overwrite=overwrite, partitions=partitions, partition=partition,
  File "/Users/tongyifan/PycharmProjects/pyodps_debug/venv/lib/python3.8/site-packages/odps/df/backends/utils.py", line 251, in write_table
    _write_table_no_partitions(frame, table, ui, cast=cast, overwrite=overwrite, partition=partition,
  File "/Users/tongyifan/PycharmProjects/pyodps_debug/venv/lib/python3.8/site-packages/odps/df/backends/utils.py", line 209, in _write_table_no_partitions
    writer.write(gen())
  File "/Users/tongyifan/PycharmProjects/pyodps_debug/venv/lib/python3.8/site-packages/odps/models/table.py", line 360, in __exit__
    self.close()
  File "/Users/tongyifan/PycharmProjects/pyodps_debug/venv/lib/python3.8/site-packages/odps/models/table.py", line 917, in close
    super(_TableWriter, self).close()
  File "/Users/tongyifan/PycharmProjects/pyodps_debug/venv/lib/python3.8/site-packages/odps/models/table.py", line 339, in close
    block_info.writer.close()
  File "/Users/tongyifan/PycharmProjects/pyodps_debug/venv/lib/python3.8/site-packages/odps/tunnel/io/writer.py", line 360, in close
    self._req_io.close()
  File "/Users/tongyifan/PycharmProjects/pyodps_debug/venv/lib/python3.8/site-packages/odps/rest.py", line 110, in close
    self.__exit__(None, None, None)
  File "/Users/tongyifan/PycharmProjects/pyodps_debug/venv/lib/python3.8/site-packages/odps/rest.py", line 118, in __exit__
    self._ctx.__exit__(*exc_info)
  File "/opt/homebrew/opt/python@3.8/Frameworks/Python.framework/Versions/3.8/lib/python3.8/contextlib.py", line 120, in __exit__
    next(self.gen)
  File "/Users/tongyifan/PycharmProjects/pyodps_debug/venv/lib/python3.8/site-packages/odps/rest.py", line 233, in _request
    yield writer
  File "/Users/tongyifan/PycharmProjects/pyodps_debug/venv/lib/python3.8/site-packages/odps/lib/requests/sessions.py", line 352, in __exit__
    self._ctx.__exit__(*exc_info)
  File "/opt/homebrew/opt/python@3.8/Frameworks/Python.framework/Versions/3.8/lib/python3.8/contextlib.py", line 120, in __exit__
    next(self.gen)
  File "/Users/tongyifan/PycharmProjects/pyodps_debug/venv/lib/python3.8/site-packages/odps/lib/requests/sessions.py", line 723, in _send
    yield writer
  File "/opt/homebrew/opt/python@3.8/Frameworks/Python.framework/Versions/3.8/lib/python3.8/contextlib.py", line 120, in __exit__
    next(self.gen)
  File "/Users/tongyifan/PycharmProjects/pyodps_debug/venv/lib/python3.8/site-packages/odps/lib/requests/adapters.py", line 554, in send_with_fileio
    r = low_conn.getresponse()
  File "/Users/tongyifan/PycharmProjects/pyodps_debug/venv/lib/python3.8/site-packages/urllib3/connection.py", line 440, in getresponse
    raise ResponseNotReady()
http.client.ResponseNotReady

Process finished with exit code 1

@wjsi
Copy link
Collaborator

wjsi commented May 18, 2023 •
edited
Loading

Verified to be an issue of urllib3 2.0.x. Please use an earlier release of urllib3.

@tongyifan
Copy link
Author

LGTM, v0.11.4.post0 works fine.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@wjsi @tongyifan