Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

make file storage unsupport streaming query #478

Merged

Conversation

lizhou1111
Copy link
Contributor

@lizhou1111 lizhou1111 commented Jan 11, 2024

PR checklist:

  • Did you run ClangFormat ?
  • Did you separate headers to a different section in existing community code base ?
  • Did you surround proton: starts/ends for new code in existing community code base ?

Please write user-readable short description of the changes:
this closed #469
Originnal file storage supports streaming query, but the global aggregation is triggered with an interval.

SELECT
  count()
FROM
  file('/home/timeplus/lijianan/ssd/protondata/server_data/user_files/data', 'CSV', 'Time string, Exchange string, Symbol string, Sale_Condition string, Trade_Volume uint32, Trade_Price decimal(20,6), Trade_Stop_Stock_Indicator string, Trade_Correction_Indicator string, Sequence_Number uint32, Trade_Id uint64, Source_of_Trade string, Trade_Reporting_Facility string, Participant_Timestamp string, Trade_Reporting_Facility_TRF_Timestamp string, Trade_Through_Exempt_Indicator uint8')
SETTINGS
  format_csv_delimiter = '|', input_format_parallel_parsing = 0

Query id: cc2652fd-6025-4772-8bc5-e75597d4e087

Connecting to localhost:8463 as user default.
Connected to proton server version 1.3.31 revision 54459.

┌─explain─────────────────────────┐
│ (Expression)                    │
│ ExpressionTransform             │
│   (StreamingAggregating)        │
│   GlobalAggregatingTransform    │
│     (Expression)                │
│     ExpressionTransform         │
│       (WatermarkStep)           │
│       WatermarkStamperTransform │
│         (ReadFromStorage)       │
│         File 01              │
└─────────────────────────────────┘

@lizhou1111 lizhou1111 marked this pull request as ready for review January 11, 2024 02:10
@lizhou1111 lizhou1111 self-assigned this Jan 11, 2024
@lizhou1111 lizhou1111 requested a review from yl-lisen January 11, 2024 02:10
Copy link
Collaborator

@yl-lisen yl-lisen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@yl-lisen yl-lisen merged commit f56f58f into develop Jan 16, 2024
21 checks passed
@yl-lisen yl-lisen deleted the bugfix/issue-469-make-file-storage-unsupport-streaming-query branch January 16, 2024 13:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

count() on file stream returns different result than view on it
2 participants