Pandas FutureWarning - concatenation with empty entries is deprecated

[clazarus]

Describe the bug

I am running a a query with multiple subqueries, multiple GROUP BY and multiple JOIN operations and when using client.query_df I get the following warning:

[.../python3.12/site-packages/clickhouse_connect/driver/npquery.py:104): 

FutureWarning: The behavior of array concatenation with empty entries is deprecated. 
In a future version, this will no longer exclude empty items when determining the result dtype. 
To retain the old behavior, exclude the empty entries before the concat operation.
  new_df_series.append(pd.concat([pd.Series(piece, copy=False) for piece in c],

Steps to reproduce

1. Unable to provide further details unless I share my query. Let me know if more details are needed.

Expected behaviour

Same result but no warning.

clickhouse-connect and/or ClickHouse server logs

Configuration

Environment

• clickhouse-connect version: 0.7.8
• Python version: 3.12.2
• Operating system: macOS and linux (happens both locally and in prod)
• Pandas version: '2.2.2'

ClickHouse server

• ClickHouse Server version: 24.2 (hosted with you)

I understand I am not providing sufficient details to reproduce this error. Please let me know if you require an example and I can try to cook one up. I think the source if relatively easy to identify in

clickhouse-connect/clickhouse_connect/driver/npquery.py

Line 104 in 457533d

new_df_series.append(pd.concat([pd.Series(piece, copy=False) for piece in c],

[genzgd]

Thanks for the report. I think this is a fairly easy fix but a small reproducible example would be very helpful for creating the related test case.

[clazarus]

Thank you very much for fixing this!

[genzgd]

I was too lazy to create a test case, can you confirm that 0.7.9 fixes the issue?

[clazarus]

I was too lazy to create a test case, can you confirm that 0.7.9 fixes the issue?

I can confirm that the issue is fixed on my end. Again sorry to not provide a reproducible example.

[manikandan96372]

I am facing "No objects to concatenate" error even after using the latest version where this is fixed, any idea how ? it exactly points to npquery.py for the error

[genzgd]

Is it possible you're getting a purely empty result set?

[manikandan96372]

It is possible, but with the latest fix that should get handled right ?

[genzgd]

@manikandan96372 No, I think that's a slightly different scenario. I'm working on a small release which has a possible fix for what you are seeing. Is your error just a warning in the logs or does it actually throw an exception?

[manikandan96372]

It throws an exception actually, ValueError: No objects to concatenate @genzgd

[genzgd]

@manikandan96372 Please try 0.7.10 and see if it fixes the issue.

[manikandan96372]

@genzgd I have upgraded to the latest version and haven't faced the same error again for the last few hours, thank you so much for the fix ! great work !