clickhouse.idle does not honour ConnMaxLifetime ?

[moming00]

Hello, was looking at the clickhouse.go from v2.7.0, and realized that conn.close() will never be called when the connection is always in idle and no further call to acquire.
For example, I have a program ingesting data to ClickHouse based on the following configuration, and at some stage, there is no further write operation required (no further input) and the program is still running in idle, then from my understanding the 5 conns will never be release untill futher call of release. And I would expect the connections be closed when reaching ConnMaxLifetime .

    clickhouse.Options{
	Addr: []string{"127.0.0.1:9000"},
	Auth: clickhouse.Auth{
		Database: "default",
		Username: "default",
		//Password: "ClickHouse",
	},
	//Debug:           true,
	DialTimeout:     time.Second,
	MaxOpenConns:    5,
	MaxIdleConns:    5,
	ConnMaxLifetime: time.Minute * 10,
	Compression: &clickhouse.Compression{
		Method: clickhouse.CompressionLZ4,
	},
	BlockBufferSize:      100,
	MaxCompressionBuffer: maxCompressionBuffer,
}

Should there be any concern about the ClickHouse performance, as from my knowledge the MaxOpenConns will be counting into max-concurrent-queries and may slow down queries from other session.

Please correct me if I misunderstood anything, thanks.

[jkaflik]

@moming00, thanks for reporting. I will be looking into this today.

[jkaflik]

@moming00

Should there be any concern about the ClickHouse performance, as from my knowledge the MaxOpenConns will be counting into max-concurrent-queries and may slow down queries from other session.

Could you elaborate on this? Did you find it in the doc or observe?

[jkaflik]

@moming00 please see draft PR - this should solve the issue: #945

[moming00]

@moming00

Should there be any concern about the ClickHouse performance, as from my knowledge the MaxOpenConns will be counting into max-concurrent-queries and may slow down queries from other session.

Could you elaborate on this? Did you find it in the doc or observe?

I was only saying based on the attached link which come from the official clickhouse doc.

[jkaflik]

@moming00 I don't think a number of idle connections has an effect on max-concurrent-queries, since there is no query running.

[moming00]

@moming00 I don't think a number of idle connections has an effect on max-concurrent-queries, since there is no query running.

but the connection is there, I believe new connection won't be able to be created then.

[jkaflik]

but the connection is there, I believe new connection won't be able to be created then.

if you refer to a bug reported by you, it's true. I refer to the max-concurrent-queries setting. Didn't you mean max_connections setting?

[moming00]

but the connection is there, I believe new connection won't be able to be created then.

if you refer to a bug reported by you, it's true. I refer to the max-concurrent-queries setting. Didn't you mean max_connections setting?

yup was also refering to the max-concurrent-queries setting, thought once the total connection count reached the default 100 limit, any new query won't be processed.