Add chunked request processing back into the Go client v2

[jgeiger]

Feature Request

Proposal: Add chunked request processing back into v2

Current behavior: Requests only return the first 10000 rows of a query

Desired behavior: Return all rows for a query that includes more than 10000 rows

Use case: We're attempting to use InfluxDB as an event store, but we have series that have more than 10000 entries. We allow people to download the entire history and right now I need to run multiple queries with LIMIT and OFFSET.

Doing a single query with chunked=true and chunk_size=10000 returns 350000 rows in about 5 seconds, while the LIMIT/OFFSET takes significantly (12x) longer.

[jwilder]

The client should use chunking by default since without it you can easily OOM the server with a bad query.

As a workaround, you can disable the limit on the server via:

[http]
max-row-limit = 0

[jgeiger]

The URL generated by the client v2 doesn't include the chunked=true or chunk_size=x query parameters.

http://localhost:8086/query?db=events&epoch=u&params={}&q=SELECT time, user, name, description FROM "h-1" ORDER BY time ASC

Adding them the the params map, for NewQueryWithParameters adds them as JSON in the params query param which fails to return more than 10k rows because it doesn't properly include the params for chunking.

http://localhost:8086/query?db=events&epoch=u&params={"chunk_size":"10000","chunked":"true"}&q=SELECT time, user, name, description FROM "h-1" ORDER BY time

[jwilder]

Yes. The v2 client doesn't have chunking support right now. I agree it should support it though.