Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Enable chunked responses and modify raw queries to be streaming #2029

Closed
pauldix opened this issue Mar 20, 2015 · 0 comments
Closed

Enable chunked responses and modify raw queries to be streaming #2029

pauldix opened this issue Mar 20, 2015 · 0 comments
Assignees
@pauldix
Milestone
0.9.0

Comments

@pauldix
Copy link
Member

pauldix commented Mar 20, 2015

Raw queries should stream out results in batches. In engine.go they should be pushing rows out the results channel as batches of data comes in.

For this to work we also have to update the handler to deal with streaming results and either send them out as the come if the user is taking chunked HTTP responses, or the handler will have to put them together in memory and return once it's done.
@pauldix pauldix self-assigned this Mar 20, 2015
@pauldix pauldix added this to the 0.9.0 milestone Mar 20, 2015
pauldix added a commit that referenced this issue Mar 28, 2015
@pauldix
Add chunked responses and streaming of raw queries.
df18ea0 
Refactored query engine to have different processing pipeline for raw queries. This enables queries that have a large offset to not keep everything in memory. It also makes it so that queries against raw data that have a limit will only p
rocess up to that limit and then bail out.

Raw data queries will only read up to a certain point in the map phase before yielding to the engine for further processing.

Fixes #2029 and fixes #2030
pauldix added a commit that referenced this issue Mar 28, 2015
@pauldix
Add chunked responses and streaming of raw queries.
a12a58a 
Refactored query engine to have different processing pipeline for raw queries. This enables queries that have a large offset to not keep everything in memory. It also makes it so that queries against raw data that have a limit will only p
rocess up to that limit and then bail out.

Raw data queries will only read up to a certain point in the map phase before yielding to the engine for further processing.

Fixes #2029 and fixes #2030
@pauldix pauldix mentioned this issue Mar 28, 2015
Closed
pauldix added a commit that referenced this issue Apr 1, 2015
@pauldix @otoolep
Add chunked responses and streaming of raw queries.
12a6fcb 
Refactored query engine to have different processing pipeline for raw queries. This enables queries that have a large offset to not keep everything in memory. It also makes it so that queries against raw data that have a limit will only p
rocess up to that limit and then bail out.

Raw data queries will only read up to a certain point in the map phase before yielding to the engine for further processing.

Fixes #2029 and fixes #2030
@pauldix pauldix closed this as completed in 6c46a5c Apr 2, 2015
pauldix added a commit that referenced this issue Apr 4, 2015
@pauldix
Add chunked responses and streaming of raw queries.
958ce63 
Refactored query engine to have different processing pipeline for raw queries. This enables queries that have a large offset to not keep everything in memory. It also makes it so that queries against raw data that have a limit will only p
rocess up to that limit and then bail out.

Raw data queries will only read up to a certain point in the map phase before yielding to the engine for further processing.

Fixes #2029 and fixes #2030
pauldix added a commit that referenced this issue Apr 6, 2015
@pauldix
Add chunked responses and streaming of raw queries.
f2cca73 
Refactored query engine to have different processing pipeline for raw queries. This enables queries that have a large offset to not keep everything in memory. It also makes it so that queries against raw data that have a limit will only p
rocess up to that limit and then bail out.

Raw data queries will only read up to a certain point in the map phase before yielding to the engine for further processing.

Fixes #2029 and fixes #2030
pauldix added a commit that referenced this issue Apr 7, 2015
@pauldix
Add chunked responses and streaming of raw queries.
57a3e0a 
Refactored query engine to have different processing pipeline for raw queries. This enables queries that have a large offset to not keep everything in memory. It also makes it so that queries against raw data that have a limit will only p
rocess up to that limit and then bail out.

Raw data queries will only read up to a certain point in the map phase before yielding to the engine for further processing.

Fixes #2029 and fixes #2030
pauldix added a commit that referenced this issue Apr 9, 2015
@pauldix @otoolep
Add chunked responses and streaming of raw queries.
7db3724 
Refactored query engine to have different processing pipeline for raw queries. This enables queries that have a large offset to not keep everything in memory. It also makes it so that queries against raw data that have a limit will only p
rocess up to that limit and then bail out.

Raw data queries will only read up to a certain point in the map phase before yielding to the engine for further processing.

Fixes #2029 and fixes #2030
pauldix added a commit that referenced this issue Apr 9, 2015
@pauldix @otoolep
Add chunked responses and streaming of raw queries.
4ea1c40 
Refactored query engine to have different processing pipeline for raw queries. This enables queries that have a large offset to not keep everything in memory. It also makes it so that queries against raw data that have a limit will only p
rocess up to that limit and then bail out.

Raw data queries will only read up to a certain point in the map phase before yielding to the engine for further processing.

Fixes #2029 and fixes #2030
pauldix added a commit that referenced this issue Apr 10, 2015
@pauldix @otoolep
Add chunked responses and streaming of raw queries.
4f66251 
Refactored query engine to have different processing pipeline for raw queries. This enables queries that have a large offset to not keep everything in memory. It also makes it so that queries against raw data that have a limit will only p
rocess up to that limit and then bail out.

Raw data queries will only read up to a certain point in the map phase before yielding to the engine for further processing.

Fixes #2029 and fixes #2030
pauldix added a commit that referenced this issue Apr 10, 2015
@pauldix @otoolep
Add chunked responses and streaming of raw queries.
2a3ed63 
Refactored query engine to have different processing pipeline for raw queries. This enables queries that have a large offset to not keep everything in memory. It also makes it so that queries against raw data that have a limit will only p
rocess up to that limit and then bail out.

Raw data queries will only read up to a certain point in the map phase before yielding to the engine for further processing.

Fixes #2029 and fixes #2030
mark-rushakoff pushed a commit that referenced this issue Jan 11, 2019
@Palakp41
Merge pull request #2029 from influxdata/fix/input-id-unnecessary
1e7cadc 
Remove id prop from input component
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@pauldix pauldix

Labels
None yet
Projects
None yet
Milestone
0.9.0
Development

Successfully merging a pull request may close this issue.

Add chunked responses and streaming of raw queries influxdata/influxdb
1 participant
@pauldix