You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Description:
Before updating to version 2.1.2, we were using Fluent Bit version 1.7, and the stream processor worked perfectly. We were obtaining satisfactory results from the stream processor using SQL syntax.
However, after updating to version 2.1.2, we encountered issues with the stream processor. Upon thorough analysis and testing, we discovered that the keys used for GROUP BY operations were no longer present in the resulting logs. Nevertheless, the corresponding values were correct.
The syntax SQL we used for applying stream processing with window tumbling is this:
CREATE STREAM agg.logs WITH (tag='agg.logs') AS SELECT server, host, SUM(size_sent), SUM(body_size), COUNT(*) FROM TAG:'logs.*' WINDOW TUMBLING (15 SECOND) GROUP BY server, host;
To Reproduce
Upgrade Fluent Bit from version 1.7 to version 2.1.
Configure and execute the stream processor using the following SQL syntax for window tumbling:
[STREAM_TASK]
Name agg_stream
Exec CREATE STREAM agg.logs WITH (tag='agg.logs') AS SELECT server, host, SUM(size_sent), SUM(body_size), COUNT(*) FROM TAG:'logs.*' WINDOW TUMBLING (15 SECOND) GROUP BY server, host;
Observe the resulting logs.
Expected behavior
The resulting logs should retain the keys used for GROUP BY operations, while ensuring that the values remain accurate, even when applying window tumbling:
Actual Behavior:
The keys used for GROUP BY operations are missing in the resulting logs when applying window tumbling, although the values are correct:
[SERVICE]
flush 1
daemon off
log_level warning
HTTP_Server On
HTTP_Listen 127.0.0.1
HTTP_PORT 2020
storage.metrics on
Log_File /var/log/fluentbit.log
Streams_File /etc/fluentbit/script/stream.conf
[INPUT]
Name syslog
tag logs.acc
Path /var/log/logs_acc.sock
Unix_Perm 777
Mem_Buf_Limit 15M
Buffer_Chunk_Size 1M
[OUTPUT]
Name stdout
Match agg.logs
streams.Conf:
[STREAM_TASK]
Name agg_stream
Exec CREATE STREAM agg.logs WITH (tag='agg.logs') AS SELECT server, host, SUM(size_sent), SUM(body_size), COUNT(*) FROM TAG:'logs.*' WINDOW TUMBLING (15 SECOND) GROUP BY server, host;
Additional Information:
Fluent Bit version: 2.1.2
Operating System: Debian GNU/Linux 11 (bullseye)
Reproducibility: Yes, the issue is consistently reproducible.
Deployment: Container Based
Please let me know if there is any further information or clarification needed
The text was updated successfully, but these errors were encountered:
This bug happen to me too, but I found a workaround for that, which is use the "AS same_name_field" for every field.
So, for your example, I rewrite the query like this:
CREATE STREAM agg.logs WITH (tag='agg.logs') AS SELECT server AS server, host AS host, SUM(size_sent) AS SUM_size_sent, SUM(body_size) AS SUM_body_size, COUNT(*) AS COUNT_total FROM TAG:'logs.*' WINDOW TUMBLING (15 SECOND) GROUP BY server, host;
Bug Report
Description:
Before updating to version 2.1.2, we were using Fluent Bit version 1.7, and the stream processor worked perfectly. We were obtaining satisfactory results from the stream processor using SQL syntax.
However, after updating to version 2.1.2, we encountered issues with the stream processor. Upon thorough analysis and testing, we discovered that the keys used for GROUP BY operations were no longer present in the resulting logs. Nevertheless, the corresponding values were correct.
The syntax SQL we used for applying stream processing with window tumbling is this:
To Reproduce
Expected behavior
The resulting logs should retain the keys used for GROUP BY operations, while ensuring that the values remain accurate, even when applying window tumbling:
Actual Behavior:
The keys used for GROUP BY operations are missing in the resulting logs when applying window tumbling, although the values are correct:
Environment:
streams.Conf:
Additional Information:
Fluent Bit version: 2.1.2
Operating System: Debian GNU/Linux 11 (bullseye)
Reproducibility: Yes, the issue is consistently reproducible.
Deployment: Container Based
Please let me know if there is any further information or clarification needed
The text was updated successfully, but these errors were encountered: