Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Doesn't work insert_df with TEMPORARY TABLE #145

Closed
khmelevskiy opened this issue Mar 10, 2023 · 1 comment · Fixed by #146
Closed

Doesn't work insert_df with TEMPORARY TABLE #145

khmelevskiy opened this issue Mar 10, 2023 · 1 comment · Fixed by #146
Labels
bug Something isn't working

Comments

@khmelevskiy
Copy link

khmelevskiy commented Mar 10, 2023

Describe the bug

Method insert_df doesn't work with TEMPORARY TABLE

Error:

Problem here
For TEMPORARY TABLE we can't use some database

Steps to reproduce

  1. Fill credentials in example
  2. Run example

Expected behaviour

No error

Code example

import clickhouse_connect

clickhouse_connect_client = clickhouse_connect.get_client(
        username="",
        password="",
        host="",
        port=,
        secure=False,
        connect_timeout=10,
        query_limit=0,
        session_id=f"test_session_id",
    )
query_create_temp_table = """
CREATE TEMPORARY TABLE temp_test_table
        (
            Filed1 String,
            Filed2 String
        )"""
clickhouse_connect_client.command(query_create_temp_table)
query_insert_temp_table = (
        """ INSERT INTO temp_test_table (Filed1, Filed2) VALUES ('test1', 'test2'), ('test3', 'test4')"""
    )
print(clickhouse_connect_client.query(query_insert_temp_table).result_rows)
query_read_temp_table = """ SELECT * FROM temp_test_table"""
df = clickhouse_connect_client.query_df(query_read_temp_table)
print("@@@ 1 @@@\n", df)
clickhouse_connect_client.insert_df(df=df, table="temp_test_table")
df = clickhouse_connect_client.query_df(query_read_temp_table)
print("@@@ 2 @@@\n", df)

clickhouse-connect and/or ClickHouse server logs

clickhouse_connect.driver.exceptions.DatabaseError: :HTTPDriver for ******* returned response code 404) Code: 60. DB::Exception: Table default.temp_test_table doesn't exist. (UNKNOWN_TABLE) (version 23.2.3.17 (official build))

Configuration

Environment

  • clickhouse-connect version: 0.5.14
  • Python version: 3.9
  • Operating system: macOS

We can use self.database = None, but i think it's bad solution.

@khmelevskiy khmelevskiy added the bug Something isn't working label Mar 10, 2023
@genzgd genzgd mentioned this issue Mar 10, 2023
2 tasks
@genzgd
Copy link
Collaborator

genzgd commented Mar 10, 2023

Thanks for the detailed steps to reproduce! A fixed version v0.5.15 should be released shortly.

Even with the fix, I would consider not using temporary tables in your application:

  • Temporary tables are primarily intended for internal use in some types of query processing. According to Alexey, in most cases they don't perform any faster than a standard MergeTree and you can actually lose some performance benefits based on compression and sorting.
  • When used over HTTP connection (like ClickHouse Connect), a temporary table is automatically dropped when the session times out. The default session_timeout is 60 seconds, so if your application doesn't issue a query for that time, your temporary table could just disappear.
  • Because HTTP doesn't maintain a permanent connection, when you "close" the client or your application, the temporary table still remains for the 60 seconds, so it may unexpectedly be there when you try to recreate it (if using the same session id)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants