Importing clickhouse_connect.driverc.dataconv fails in Python 3.10 and 3.11

[georgipeev]

Describe the bug

The clickhouse_connect.driver.ctypes module includes this import on line 18:

import clickhouse_connect.driverc.dataconv as cdc

This import fails with the following error:

  File "clickhouse_connect/driverc/dataconv.pyx", line 2, in init clickhouse_connect.driverc.dataconv
ModuleNotFoundError: No module named '_testbuffer'

Failing to import the module causes later errors such as this one on query_df:

Traceback (most recent call last):
  File "/home/gpeev/venv/3.11/lib/python3.11/site-packages/clickhouse_connect/driver/transform.py", line 50, in get_block
    column = col_type.read_column(source, num_rows, context)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/gpeev/venv/3.11/lib/python3.11/site-packages/clickhouse_connect/datatypes/base.py", line 143, in read_column
    return self.read_column_data(source, num_rows, ctx)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/gpeev/venv/3.11/lib/python3.11/site-packages/clickhouse_connect/datatypes/base.py", line 158, in read_column_data
    column = self._read_column_binary(source, num_rows, ctx)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/gpeev/venv/3.11/lib/python3.11/site-packages/clickhouse_connect/datatypes/temporal.py", line 123, in _read_column_binary
    np_array = numpy_conv.read_numpy_array(source, '<u4', num_rows).astype(self.np_type)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: Argument 'buffer' has incorrect type (expected clickhouse_connect.driverc.buffer.ResponseBuffer, got ResponseBuffer)

Empirical tests show that the above problem is very dependent on the versions of Python and clickhouse_connect:

1. Python 3.9: no import error for clickhouse_connect versions 0.6.0-0.6.15
2. Python 3.10 and 3.11: no import error for clickhouse_connect version 0.6.0-0.6.1, import error for versions 0.6.2-0.6.15

Python 3.12 was not tested, the library doesn't support it since, at least, pkg_resources no longer exists.

Steps to reproduce

1. Create a clean Python environment with Python 3.10 or Python 3.11
2. pip install clickhouse-connect==0.6.15 (or any other version 0.6.2 and higher)
3. python -c import "clickhouse_connect.driverc.dataconv as cdc"

The last command will yield the import error:

Unable to connect optimized C data functions [No module named '_testbuffer'], falling back to pure Python
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "clickhouse_connect/driverc/dataconv.pyx", line 2, in init clickhouse_connect.driverc.dataconv
ModuleNotFoundError: No module named '_testbuffer'

Expected behaviour

The import should either work correctly, or, since clickhouse_connect.driver.ctypes explicitly handles possible ImportErrors when importing it and tries to continue, subsequent library behavior should still be correct - i.e., query_df should not fail with the aforementioned TypeError: Argument 'buffer' has incorrect type (expected clickhouse_connect.driverc.buffer.ResponseBuffer, got ResponseBuffer).

Configuration

Environment

• clickhouse-connect version: >=0.6.2
• Python version: 3.10 or 3.11 (but not 3.9)
• Operating system: RHEL 8.6

[genzgd]

Again thanks for the report! As always I appreciate the extensive testing at the margins.

This might be specific to Redhat, since Ubuntu tests and builds seem to work fine. I'm not sure what you mean with the pkg_resources comment, since that dependency was removed in v0.6.5.

In any case I think I have a fix that will be released shortly. PyCharm automatically imported from a "private" SDK package and I didn't catch it.

[georgipeev]

As usual, thanks for the prompt response. Re Python 3.12 - that might be my bad, I probably only tested 0.6.2 with that Python version.

[genzgd]

RHEL imports should work correctly with the latest v0.6.16 release.

[georgipeev]

0.6.16 version works fine. Thanks!