Small memory leaks when using VAD API (whisper_vad_init_with_params / whisper_vad_free)

[michaela-tubi]

Environment

OS: Ubuntu 22.04 (Linux)
Whisper versions tested: v1.7.6 and 1.8.0
Whisper build options:

cmake .. \
  -DCMAKE_BUILD_TYPE=Release \
  -DBUILD_SHARED_LIBS=OFF \
  -DWHISPER_BUILD_EXAMPLES=OFF \
  -DWHISPER_BUILD_TESTS=OFF \
  -DGGML_NATIVE=OFF; \
make -j${jobs} && \
make install

Minimal Reproduction

cpp program

// vad_leak_repro.cpp
#include "whisper.h"

int main() {
    auto params = whisper_vad_default_context_params();
    params.n_threads = 1;
    params.use_gpu = false;

    const char *model_path = "ggml-silero-v5.1.2.bin";

    auto vctx =
        whisper_vad_init_from_file_with_params(model_path, params);

    whisper_vad_free(vctx);
    return 0; 
}

Compile

c++ -O0 -g -fsanitize=address -fno-omit-frame-pointer   -I/usr/local/include   vad_leak_repro.cpp -o vad_leak_repro   -L/usr/local/lib   -fopenmp   -Wl,--start-group     -lwhisper -lggml -lggml-base -lggml-cpu   -Wl,--end-group   -ldl -lpthread -lm

Run

./vad_leak_repro

Results

AddressSanitizer reports several small leaks after a single init/free cycle of the VAD context:

whisper_vad_init_from_file_with_params: loading VAD model from 'ggml-silero-v5.1.2.bin'
whisper_vad_init_with_params: model type: silero-16k
whisper_vad_init_with_params: model version: 5.1.2
whisper_vad_init_with_params: n_encoder_layers = 4
whisper_vad_init_with_params: encoder_in_channels[0] = 129
whisper_vad_init_with_params: encoder_in_channels[1] = 128
whisper_vad_init_with_params: encoder_in_channels[2] = 64
whisper_vad_init_with_params: encoder_in_channels[3] = 64
whisper_vad_init_with_params: encoder_out_channels[0] = 128
whisper_vad_init_with_params: encoder_out_channels[1] = 64
whisper_vad_init_with_params: encoder_out_channels[2] = 64
whisper_vad_init_with_params: encoder_out_channels[3] = 128
whisper_vad_init_with_params: lstm_input_size = 128
whisper_vad_init_with_params: lstm_hidden_size = 128
whisper_vad_init_with_params: final_conv_in = 128
whisper_vad_init_with_params: final_conv_out = 1
whisper_vad_init_with_params:          CPU total size =     0.88 MB
whisper_vad_init_with_params: model size    =    0.88 MB
whisper_backend_init_gpu: no GPU found
whisper_vad_init_context: compute buffer (VAD)   =    1.60 MB

=================================================================
==555==ERROR: LeakSanitizer: detected memory leaks

Direct leak of 104 byte(s) in 1 object(s) allocated from:
    #0 0xaaaab89cd13c in operator new(unsigned long) (vad_leak_repro+0x11d13c) (BuildId: d3cfec8067e26e683c7c41f2aaf9fb6ce9b26041)
    #1 0xaaaab8a2c0b0 in ggml_backend_cpu_buffer_type_alloc_buffer(ggml_backend_buffer_type*, unsigned long) ggml-backend.cpp
    #2 0xaaaab8a247e4 in alloc_tensor_range ggml-alloc.c
    #3 0xaaaab8a24718 in ggml_backend_alloc_ctx_tensors_from_buft (vad_leak_repro+0x174718) (BuildId: d3cfec8067e26e683c7c41f2aaf9fb6ce9b26041)
    #4 0xaaaab89d8e8c in whisper_vad_init_with_params (vad_leak_repro+0x128e8c) (BuildId: d3cfec8067e26e683c7c41f2aaf9fb6ce9b26041)
    #5 0xaaaab89d7760 in whisper_vad_init_from_file_with_params (vad_leak_repro+0x127760) (BuildId: d3cfec8067e26e683c7c41f2aaf9fb6ce9b26041)
    #6 0xaaaab89cf42c in main vad_leak_repro.cpp:12:9
    #7 0xffff9bc773f8  (/lib/aarch64-linux-gnu/libc.so.6+0x273f8) (BuildId: a10ee556ec5706640b27d9a9c346cb29e9516cec)
    #8 0xffff9bc774c8 in __libc_start_main (/lib/aarch64-linux-gnu/libc.so.6+0x274c8) (BuildId: a10ee556ec5706640b27d9a9c346cb29e9516cec)
    #9 0xaaaab88f27ec in _start (vad_leak_repro+0x427ec) (BuildId: d3cfec8067e26e683c7c41f2aaf9fb6ce9b26041)

Direct leak of 40 byte(s) in 1 object(s) allocated from:
    #0 0xaaaab8992c5c in malloc (vad_leak_repro+0xe2c5c) (BuildId: d3cfec8067e26e683c7c41f2aaf9fb6ce9b26041)
    #1 0xaaaab8a11f44 in ggml_init (vad_leak_repro+0x161f44) (BuildId: d3cfec8067e26e683c7c41f2aaf9fb6ce9b26041)
    #2 0xaaaab89d8e30 in whisper_vad_init_with_params (vad_leak_repro+0x128e30) (BuildId: d3cfec8067e26e683c7c41f2aaf9fb6ce9b26041)
    #3 0xaaaab89d7760 in whisper_vad_init_from_file_with_params (vad_leak_repro+0x127760) (BuildId: d3cfec8067e26e683c7c41f2aaf9fb6ce9b26041)
    #4 0xaaaab89cf42c in main vad_leak_repro.cpp:12:9
    #5 0xffff9bc773f8  (/lib/aarch64-linux-gnu/libc.so.6+0x273f8) (BuildId: a10ee556ec5706640b27d9a9c346cb29e9516cec)
    #6 0xffff9bc774c8 in __libc_start_main (/lib/aarch64-linux-gnu/libc.so.6+0x274c8) (BuildId: a10ee556ec5706640b27d9a9c346cb29e9516cec)
    #7 0xaaaab88f27ec in _start (vad_leak_repro+0x427ec) (BuildId: d3cfec8067e26e683c7c41f2aaf9fb6ce9b26041)

Direct leak of 16 byte(s) in 1 object(s) allocated from:
    #0 0xaaaab89cd260 in operator new[](unsigned long) (vad_leak_repro+0x11d260) (BuildId: d3cfec8067e26e683c7c41f2aaf9fb6ce9b26041)
    #1 0xaaaab89d8108 in whisper_vad_init_with_params (vad_leak_repro+0x128108) (BuildId: d3cfec8067e26e683c7c41f2aaf9fb6ce9b26041)
    #2 0xaaaab89d7760 in whisper_vad_init_from_file_with_params (vad_leak_repro+0x127760) (BuildId: d3cfec8067e26e683c7c41f2aaf9fb6ce9b26041)
    #3 0xaaaab89cf42c in main vad_leak_repro.cpp:12:9
    #4 0xffff9bc773f8  (/lib/aarch64-linux-gnu/libc.so.6+0x273f8) (BuildId: a10ee556ec5706640b27d9a9c346cb29e9516cec)
    #5 0xffff9bc774c8 in __libc_start_main (/lib/aarch64-linux-gnu/libc.so.6+0x274c8) (BuildId: a10ee556ec5706640b27d9a9c346cb29e9516cec)
    #6 0xaaaab88f27ec in _start (vad_leak_repro+0x427ec) (BuildId: d3cfec8067e26e683c7c41f2aaf9fb6ce9b26041)

Direct leak of 16 byte(s) in 1 object(s) allocated from:
    #0 0xaaaab89cd260 in operator new[](unsigned long) (vad_leak_repro+0x11d260) (BuildId: d3cfec8067e26e683c7c41f2aaf9fb6ce9b26041)
    #1 0xaaaab89d8114 in whisper_vad_init_with_params (vad_leak_repro+0x128114) (BuildId: d3cfec8067e26e683c7c41f2aaf9fb6ce9b26041)
    #2 0xaaaab89d7760 in whisper_vad_init_from_file_with_params (vad_leak_repro+0x127760) (BuildId: d3cfec8067e26e683c7c41f2aaf9fb6ce9b26041)
    #3 0xaaaab89cf42c in main vad_leak_repro.cpp:12:9
    #4 0xffff9bc773f8  (/lib/aarch64-linux-gnu/libc.so.6+0x273f8) (BuildId: a10ee556ec5706640b27d9a9c346cb29e9516cec)
    #5 0xffff9bc774c8 in __libc_start_main (/lib/aarch64-linux-gnu/libc.so.6+0x274c8) (BuildId: a10ee556ec5706640b27d9a9c346cb29e9516cec)
    #6 0xaaaab88f27ec in _start (vad_leak_repro+0x427ec) (BuildId: d3cfec8067e26e683c7c41f2aaf9fb6ce9b26041)

Direct leak of 16 byte(s) in 1 object(s) allocated from:
    #0 0xaaaab89cd260 in operator new[](unsigned long) (vad_leak_repro+0x11d260) (BuildId: d3cfec8067e26e683c7c41f2aaf9fb6ce9b26041)
    #1 0xaaaab89d8120 in whisper_vad_init_with_params (vad_leak_repro+0x128120) (BuildId: d3cfec8067e26e683c7c41f2aaf9fb6ce9b26041)
    #2 0xaaaab89d7760 in whisper_vad_init_from_file_with_params (vad_leak_repro+0x127760) (BuildId: d3cfec8067e26e683c7c41f2aaf9fb6ce9b26041)
    #3 0xaaaab89cf42c in main vad_leak_repro.cpp:12:9
    #4 0xffff9bc773f8  (/lib/aarch64-linux-gnu/libc.so.6+0x273f8) (BuildId: a10ee556ec5706640b27d9a9c346cb29e9516cec)
    #5 0xffff9bc774c8 in __libc_start_main (/lib/aarch64-linux-gnu/libc.so.6+0x274c8) (BuildId: a10ee556ec5706640b27d9a9c346cb29e9516cec)
    #6 0xaaaab88f27ec in _start (vad_leak_repro+0x427ec) (BuildId: d3cfec8067e26e683c7c41f2aaf9fb6ce9b26041)

Indirect leak of 1024 byte(s) in 1 object(s) allocated from:
    #0 0xaaaab8993770 in posix_memalign (vad_leak_repro+0xe3770) (BuildId: d3cfec8067e26e683c7c41f2aaf9fb6ce9b26041)
    #1 0xaaaab8a10be4 in ggml_aligned_malloc (vad_leak_repro+0x160be4) (BuildId: d3cfec8067e26e683c7c41f2aaf9fb6ce9b26041)
    #2 0xaaaab8a2c0a0 in ggml_backend_cpu_buffer_type_alloc_buffer(ggml_backend_buffer_type*, unsigned long) ggml-backend.cpp
    #3 0xaaaab8a247e4 in alloc_tensor_range ggml-alloc.c
    #4 0xaaaab8a24718 in ggml_backend_alloc_ctx_tensors_from_buft (vad_leak_repro+0x174718) (BuildId: d3cfec8067e26e683c7c41f2aaf9fb6ce9b26041)
    #5 0xaaaab89d8e8c in whisper_vad_init_with_params (vad_leak_repro+0x128e8c) (BuildId: d3cfec8067e26e683c7c41f2aaf9fb6ce9b26041)
    #6 0xaaaab89d7760 in whisper_vad_init_from_file_with_params (vad_leak_repro+0x127760) (BuildId: d3cfec8067e26e683c7c41f2aaf9fb6ce9b26041)
    #7 0xaaaab89cf42c in main vad_leak_repro.cpp:12:9
    #8 0xffff9bc773f8  (/lib/aarch64-linux-gnu/libc.so.6+0x273f8) (BuildId: a10ee556ec5706640b27d9a9c346cb29e9516cec)
    #9 0xffff9bc774c8 in __libc_start_main (/lib/aarch64-linux-gnu/libc.so.6+0x274c8) (BuildId: a10ee556ec5706640b27d9a9c346cb29e9516cec)
    #10 0xaaaab88f27ec in _start (vad_leak_repro+0x427ec) (BuildId: d3cfec8067e26e683c7c41f2aaf9fb6ce9b26041)

SUMMARY: AddressSanitizer: 1216 byte(s) leaked in 6 allocation(s).

Likely sources

• whisper_vad_context.buffer
  I do not see a corresponding free for this member in whisper_vad_free().
• Small ggml_init context used for LSTM state (h_state, c_state)
  A ggml_context is created in whisper_vad_init_context() to hold the LSTM states. It is not stored anywhere to later be freed.
• Per-model hparams arrays (encoder_in_channels, encoder_out_channels, kernel_sizes)
  These are filled during whisper_vad_init_with_params() using new[]. I don’t see corresponding delete[] calls in the free path.

Impact

Small but persistent leaks in applications that frequently create/destroy a VAD context

[danbev]

@michaela-tubi Thanks for the detailed report!
I'm to blame for all of these I'm afraid and I've opened #3453 to address them.

[michaela-tubi]

Thanks for the quick response and fix!