in_calyptia_fleet: fix memory leaks.

[pwhelan]

Summary

Fix the majority of the memory leaks in the calyptia fleet code.

Here is a run of fluent-bit with the patch under valgrind:

valgrind --leak-check=full ./bin/fluent-bit -c fleet.conf
==85735== Memcheck, a memory error detector
==85735== Copyright (C) 2002-2022, and GNU GPL'd, by Julian Seward et al.
==85735== Using Valgrind-3.22.0 and LibVEX; rerun with -h for copyright info
==85735== Command: ./bin/fluent-bit -c fleet.conf
==85735==
Fluent Bit v2.2.1
* Copyright (C) 2015-2023 The Fluent Bit Authors
* Fluent Bit is a CNCF sub-project under the umbrella of Fluentd
* https://fluentbit.io

[2023/12/11 21:33:06] [ info] [fluent bit] version=2.2.1, commit=469652f625, pid=85735
[2023/12/11 21:33:06] [ info] [custom:calyptia:calyptia.0] custom initialized!
[2023/12/11 21:33:06] [ info] [storage] ver=1.5.1, type=memory, sync=normal, checksum=off, max_chunks_up=128
[2023/12/11 21:33:06] [ info] [cmetrics] version=0.6.5
[2023/12/11 21:33:06] [ info] [ctraces ] version=0.3.1
[2023/12/11 21:33:06] [ info] [input:fluentbit_metrics:fluentbit_metrics.0] initializing
[2023/12/11 21:33:06] [ info] [input:fluentbit_metrics:fluentbit_metrics.0] storage_strategy='memory' (memory only)
[2023/12/11 21:33:06] [ info] [input:calyptia_fleet:calyptia_fleet.1] initializing
[2023/12/11 21:33:06] [ info] [input:calyptia_fleet:calyptia_fleet.1] storage_strategy='memory' (memory only)
[2023/12/11 21:33:06] [ info] [input:calyptia_fleet:calyptia_fleet.1] initializing calyptia fleet input.
[2023/12/11 21:33:06] [ warn] [read_glob] glob: [/tmp/calyptia-fleet/99d70f0171de7aec9b8ae68b1f6e32b7dd65763b0b73dd603ed95a112b0bf8ee/fleet-script-file/*.conf] no match
[2023/12/11 21:33:06] [error] [input:calyptia_fleet:calyptia_fleet.1] unable to read fleet directory for conf files: /tmp/calyptia-fleet/99d70f0171de7aec9b8ae68b1f6e32b7dd65763b0b73dd603ed95a112b0bf8ee/fleet-script-file/*.conf
[2023/12/11 21:33:06] [ warn] [input:calyptia_fleet:calyptia_fleet.1] unable to find latest configuration file
[2023/12/11 21:33:06] [ info] [sp] stream processor started
[2023/12/11 21:33:22] [ info] [input:calyptia_fleet:calyptia_fleet.1] loading configuration from /tmp/calyptia-fleet/99d70f0171de7aec9b8ae68b1f6e32b7dd65763b0b73dd603ed95a112b0bf8ee/fleet-script-file/new.conf.
[2023/12/11 21:33:22] [ info] [input:calyptia_fleet:calyptia_fleet.1] changing to config dir: /tmp/calyptia-fleet/99d70f0171de7aec9b8ae68b1f6e32b7dd65763b0b73dd603ed95a112b0bf8ee/fleet-script-file/1700004854
[2023/12/11 21:33:22] [ info] [input:calyptia_fleet:calyptia_fleet.1] changing to config dir: /tmp/calyptia-fleet/99d70f0171de7aec9b8ae68b1f6e32b7dd65763b0b73dd603ed95a112b0bf8ee/fleet-script-file/1700004854
[2023/12/11 21:33:27] [engine] caught signal (SIGHUP)
[2023/12/11 21:33:27] [ info] reloading instance pid=85735 tid=0x5a24900
[2023/12/11 21:33:27] [ info] [reload] stop everything of the old context
[2023/12/11 21:33:27] [ warn] [engine] service will shutdown when all remaining tasks are flushed
[2023/12/11 21:33:27] [ info] [input] pausing fluentbit_metrics.0
[2023/12/11 21:33:27] [ info] [input] pausing calyptia_fleet.1
[2023/12/11 21:33:28] [ info] [engine] service has stopped (0 pending tasks)
[2023/12/11 21:33:28] [ info] [input] pausing fluentbit_metrics.0
[2023/12/11 21:33:28] [ info] [input] pausing calyptia_fleet.1
[2023/12/11 21:33:28] [ info] [reload] start everything
[2023/12/11 21:33:28] [ info] [fluent bit] version=2.2.1, commit=469652f625, pid=85735
[2023/12/11 21:33:28] [ info] [custom:calyptia:calyptia.0] custom initialized!
[2023/12/11 21:33:28] [ info] [storage] ver=1.5.1, type=memory, sync=normal, checksum=off, max_chunks_up=128
[2023/12/11 21:33:28] [ info] [cmetrics] version=0.6.5
[2023/12/11 21:33:28] [ info] [ctraces ] version=0.3.1
[2023/12/11 21:33:28] [ info] [input:dummy:dummy.0] initializing
[2023/12/11 21:33:28] [ info] [input:dummy:dummy.0] storage_strategy='memory' (memory only)
[2023/12/11 21:33:28] [ info] [input:fluentbit_metrics:fluentbit_metrics.1] initializing
[2023/12/11 21:33:28] [ info] [input:fluentbit_metrics:fluentbit_metrics.1] storage_strategy='memory' (memory only)
[2023/12/11 21:33:28] [ info] [output:stdout:stdout.0] worker #0 started
[2023/12/11 21:33:28] [ info] [input:calyptia_fleet:calyptia_fleet.2] initializing
[2023/12/11 21:33:28] [ info] [input:calyptia_fleet:calyptia_fleet.2] storage_strategy='memory' (memory only)
[2023/12/11 21:33:28] [ info] [input:calyptia_fleet:calyptia_fleet.2] initializing calyptia fleet input.
[2023/12/11 21:33:29] [ info] [output:calyptia:calyptia.1] connected to Calyptia, agent_id='3367b469-eb0d-42e3-8043-976222296359'
[2023/12/11 21:33:29] [ info] [sp] stream processor started
[0] dummy: [[1702341209.541482612, {}], {"message"=>"unscripted #13", "powered_by"=>"calyptia"}]
[0] dummy: [[1702341210.543073821, {}], {"message"=>"unscripted #13", "powered_by"=>"calyptia"}]
[0] dummy: [[1702341211.539730251, {}], {"message"=>"unscripted #13", "powered_by"=>"calyptia"}]
[0] dummy: [[1702341212.539373817, {}], {"message"=>"unscripted #13", "powered_by"=>"calyptia"}]
[0] dummy: [[1702341213.546992698, {}], {"message"=>"unscripted #13", "powered_by"=>"calyptia"}]
==85735== Thread 2 flb-pipeline:
==85735== Conditional jump or move depends on uninitialised value(s)
==85735==    at 0x21B2D3: output_pre_cb_flush (flb_output.h:584)
==85735==    by 0xE68626: co_init (amd64.c:117)
==85735==
[0] dummy: [[1702341214.544351485, {}], {"message"=>"unscripted #13", "powered_by"=>"calyptia"}]
[0] dummy: [[1702341215.539688866, {}], {"message"=>"unscripted #13", "powered_by"=>"calyptia"}]
[0] dummy: [[1702341216.539253478, {}], {"message"=>"unscripted #13", "powered_by"=>"calyptia"}]
[0] dummy: [[1702341217.539311886, {}], {"message"=>"unscripted #13", "powered_by"=>"calyptia"}]
^C[2023/12/11 21:33:39] [engine] caught signal (SIGINT)
[2023/12/11 21:33:39] [ warn] [engine] service will shutdown in max 5 seconds
[0] dummy: [[1702341218.540250410, {}], {"message"=>"unscripted #13", "powered_by"=>"calyptia"}]
[2023/12/11 21:33:39] [ info] [input] pausing dummy.0
[2023/12/11 21:33:39] [ info] [input] pausing fluentbit_metrics.1
[2023/12/11 21:33:39] [ info] [input] pausing calyptia_fleet.2
[2023/12/11 21:33:39] [ info] [engine] service has stopped (0 pending tasks)
[2023/12/11 21:33:39] [ info] [input] pausing dummy.0
[2023/12/11 21:33:39] [ info] [input] pausing fluentbit_metrics.1
[2023/12/11 21:33:39] [ info] [input] pausing calyptia_fleet.2
[2023/12/11 21:33:39] [ info] [output:stdout:stdout.0] thread worker #0 stopping...
[2023/12/11 21:33:39] [ info] [output:stdout:stdout.0] thread worker #0 stopped
==85735==
==85735== HEAP SUMMARY:
==85735==     in use at exit: 9,369 bytes in 14 blocks
==85735==   total heap usage: 151,830 allocs, 151,816 frees, 20,554,606 bytes allocated
==85735==
==85735== Thread 1:
==85735== 27 bytes in 1 blocks are definitely lost in loss record 1 of 6
==85735==    at 0x4841828: malloc (vg_replace_malloc.c:442)
==85735==    by 0x1F250C: flb_malloc (flb_mem.h:80)
==85735==    by 0x1F26EB: sds_alloc (flb_sds.c:41)
==85735==    by 0x1F276E: flb_sds_create_len (flb_sds.c:62)
==85735==    by 0x1F2810: flb_sds_create (flb_sds.c:88)
==85735==    by 0x1CC18E: flb_service_conf_path_set (fluent-bit.c:736)
==85735==    by 0x1CC201: service_configure (fluent-bit.c:762)
==85735==    by 0x1CCBFD: flb_main (fluent-bit.c:1298)
==85735==    by 0x1CCEE9: main (fluent-bit.c:1456)
==85735==
==85735== 4,113 bytes in 1 blocks are definitely lost in loss record 6 of 6
==85735==    at 0x4841828: malloc (vg_replace_malloc.c:442)
==85735==    by 0x1F250C: flb_malloc (flb_mem.h:80)
==85735==    by 0x1F26EB: sds_alloc (flb_sds.c:41)
==85735==    by 0x1F282A: flb_sds_create_size (flb_sds.c:93)
==85735==    by 0x4CA5B2: generate_base_fleet_directory (in_calyptia_fleet.c:237)
==85735==    by 0x4CA692: fleet_config_filename (in_calyptia_fleet.c:260)
==85735==    by 0x4CA74C: time_fleet_config_filename (in_calyptia_fleet.c:282)
==85735==    by 0x4CE410: get_calyptia_fleet_config (in_calyptia_fleet.c:1789)
==85735==    by 0x4CE768: in_calyptia_fleet_collect (in_calyptia_fleet.c:1841)
==85735==    by 0x1FF969: input_pre_cb_collect (flb_input.h:517)
==85735==    by 0xE68626: co_init (amd64.c:117)
==85735==
==85735== LEAK SUMMARY:
==85735==    definitely lost: 4,140 bytes in 2 blocks
==85735==    indirectly lost: 0 bytes in 0 blocks
==85735==      possibly lost: 0 bytes in 0 blocks
==85735==    still reachable: 5,229 bytes in 12 blocks
==85735==         suppressed: 0 bytes in 0 blocks
==85735== Reachable blocks (those to which a pointer was found) are not shown.
==85735== To see them, rerun with: --leak-check=full --show-leak-kinds=all
==85735==
==85735== Use --track-origins=yes to see where uninitialised values come from
==85735== For lists of detected and suppressed errors, rerun with: -s
==85735== ERROR SUMMARY: 3 errors from 3 contexts (suppressed: 0 from 0)

The last memory leak would require freeing the configuration name upon reload or when exiting fluent-bit.

---

Enter [N/A] in the box, if an item is not applicable to your change.

Testing
Before we can approve your change; please submit the following in a comment:

• Example configuration file for the change
• Debug log output from testing the change

• Attached Valgrind output that shows no leaks or memory corruption was found

If this is a change to packaging of containers or native binaries then please confirm it works for all targets.

• Run local packaging test showing all targets (including any new ones) build.
• Set ok-package-test label to test for all targets (requires maintainer to do).

Documentation

• Documentation required for this feature

Backporting

• Backport to latest stable release.

---

Fluent Bit is licensed under Apache 2.0, by submitting this pull request I understand that this code will be released under the terms of that license.