Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix memory leak on test code #1776

Merged
merged 1 commit into from
Jun 28, 2024

Conversation

heemin32
Copy link
Collaborator

Description

Fix memory leak on test code

Issues Resolved

#1772

Check List

  • New functionality includes testing.
    • All tests pass
  • New functionality has been documented.
    • New functionality has javadoc added
  • Commits are signed as per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

==30963== HEAP SUMMARY:
==30963==     in use at exit: 394,772 bytes in 41 blocks
==30963==   total heap usage: 589,824 allocs, 589,783 frees, 983,712,368 bytes allocated
==30963== 
==30963== 72 bytes in 1 blocks are definitely lost in loss record 8 of 33
==30963==    at 0x48657B8: operator new(unsigned long) (in /usr/libexec/valgrind/vgpreload_memcheck-arm64-linux.so)
==30963==    by 0x1F08D7: FaissIsSharedIndexStateRequired_BasicAssertions_Test::TestBody() (in /workspace/heemin/k-NN/jni/bin/jni_test)
==30963==    by 0x7BBB8B: void testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) (in /workspace/heemin/k-NN/jni/bin/jni_test)
==30963==    by 0x7A9F5B: testing::Test::Run() [clone .part.0] (in /workspace/heemin/k-NN/jni/bin/jni_test)
==30963==    by 0x7AA52F: testing::TestInfo::Run() (in /workspace/heemin/k-NN/jni/bin/jni_test)
==30963==    by 0x7AB43F: testing::TestSuite::Run() [clone .part.0] (in /workspace/heemin/k-NN/jni/bin/jni_test)
==30963==    by 0x7B2A0F: testing::internal::UnitTestImpl::RunAllTests() (in /workspace/heemin/k-NN/jni/bin/jni_test)
==30963==    by 0x7AA647: testing::UnitTest::Run() (in /workspace/heemin/k-NN/jni/bin/jni_test)
==30963==    by 0x1EDD7F: main (in /workspace/heemin/k-NN/jni/bin/jni_test)
==30963== 
==30963== 72 bytes in 1 blocks are definitely lost in loss record 9 of 33
==30963==    at 0x48657B8: operator new(unsigned long) (in /usr/libexec/valgrind/vgpreload_memcheck-arm64-linux.so)
==30963==    by 0x1F0917: FaissIsSharedIndexStateRequired_BasicAssertions_Test::TestBody() (in /workspace/heemin/k-NN/jni/bin/jni_test)
==30963==    by 0x7BBB8B: void testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) (in /workspace/heemin/k-NN/jni/bin/jni_test)
==30963==    by 0x7A9F5B: testing::Test::Run() [clone .part.0] (in /workspace/heemin/k-NN/jni/bin/jni_test)
==30963==    by 0x7AA52F: testing::TestInfo::Run() (in /workspace/heemin/k-NN/jni/bin/jni_test)
==30963==    by 0x7AB43F: testing::TestSuite::Run() [clone .part.0] (in /workspace/heemin/k-NN/jni/bin/jni_test)
==30963==    by 0x7B2A0F: testing::internal::UnitTestImpl::RunAllTests() (in /workspace/heemin/k-NN/jni/bin/jni_test)
==30963==    by 0x7AA647: testing::UnitTest::Run() (in /workspace/heemin/k-NN/jni/bin/jni_test)
==30963==    by 0x1EDD7F: main (in /workspace/heemin/k-NN/jni/bin/jni_test)
==30963== 
==30963== 84 (24 direct, 60 indirect) bytes in 1 blocks are definitely lost in loss record 12 of 33
==30963==    at 0x48657B8: operator new(unsigned long) (in /usr/libexec/valgrind/vgpreload_memcheck-arm64-linux.so)
==30963==    by 0x5A6AFE7: knn_jni::commons::storeVectorData(knn_jni::JNIUtilInterface*, JNIEnv_*, long, _jobjectArray*, long) (in /workspace/heemin/k-NN/jni/release/libopensearchknn_util.so)
==30963==    by 0x28F69F: CommonsTests_BasicAssertions_Test::TestBody() (in /workspace/heemin/k-NN/jni/bin/jni_test)
==30963==    by 0x7BBB8B: void testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) (in /workspace/heemin/k-NN/jni/bin/jni_test)
==30963==    by 0x7A9F5B: testing::Test::Run() [clone .part.0] (in /workspace/heemin/k-NN/jni/bin/jni_test)
==30963==    by 0x7AA52F: testing::TestInfo::Run() (in /workspace/heemin/k-NN/jni/bin/jni_test)
==30963==    by 0x7AB43F: testing::TestSuite::Run() [clone .part.0] (in /workspace/heemin/k-NN/jni/bin/jni_test)
==30963==    by 0x7B2A0F: testing::internal::UnitTestImpl::RunAllTests() (in /workspace/heemin/k-NN/jni/bin/jni_test)
==30963==    by 0x7AA647: testing::UnitTest::Run() (in /workspace/heemin/k-NN/jni/bin/jni_test)
==30963==    by 0x1EDD7F: main (in /workspace/heemin/k-NN/jni/bin/jni_test)
==30963== 
==30963== 3,312 bytes in 9 blocks are possibly lost in loss record 22 of 33
==30963==    at 0x4869F34: calloc (in /usr/libexec/valgrind/vgpreload_memcheck-arm64-linux.so)
==30963==    by 0x4010F83: calloc (rtld-malloc.h:44)
==30963==    by 0x4010F83: allocate_dtv (dl-tls.c:375)
==30963==    by 0x4011983: _dl_allocate_tls (dl-tls.c:634)
==30963==    by 0x5FEE087: allocate_stack (allocatestack.c:430)
==30963==    by 0x5FEE087: pthread_create@@GLIBC_2.34 (pthread_create.c:647)
==30963==    by 0x5C2E873: ??? (in /usr/lib/aarch64-linux-gnu/libgomp.so.1.0.0)
==30963==    by 0x5C25E9B: GOMP_parallel (in /usr/lib/aarch64-linux-gnu/libgomp.so.1.0.0)
==30963==    by 0x44A1F7: faiss::exhaustive_L2sqr_fused_cmax_simdlib(float const*, float const*, unsigned long, unsigned long, unsigned long, faiss::Top1BlockResultHandler<faiss::CMax<float, long> >&, float const*) (in /workspace/heemin/k-NN/jni/bin/jni_test)
==30963==    by 0x406BD7: faiss::knn_L2sqr(float const*, float const*, unsigned long, unsigned long, unsigned long, unsigned long, float*, long*, float const*, faiss::IDSelector const*) (in /workspace/heemin/k-NN/jni/bin/jni_test)
==30963==    by 0x408633: faiss::knn_L2sqr(float const*, float const*, unsigned long, unsigned long, unsigned long, faiss::HeapArray<faiss::CMax<float, long> >*, float const*, faiss::IDSelector const*) (in /workspace/heemin/k-NN/jni/bin/jni_test)
==30963==    by 0x2BFA5B: faiss::IndexFlat::search(long, float const*, long, float*, long*, faiss::SearchParameters const*) const (in /workspace/heemin/k-NN/jni/bin/jni_test)
==30963==    by 0x4538D3: faiss::Clustering::train_encoded(long, unsigned char const*, faiss::Index const*, faiss::Index&, float const*) (in /workspace/heemin/k-NN/jni/bin/jni_test)
==30963==    by 0x38938B: faiss::ProductQuantizer::train(unsigned long, float const*) (in /workspace/heemin/k-NN/jni/bin/jni_test)
==30963== 
==30963== 131,960 (520 direct, 131,440 indirect) bytes in 1 blocks are definitely lost in loss record 32 of 33
==30963==    at 0x48657B8: operator new(unsigned long) (in /usr/libexec/valgrind/vgpreload_memcheck-arm64-linux.so)
==30963==    by 0x1F096F: FaissIsSharedIndexStateRequired_BasicAssertions_Test::TestBody() (in /workspace/heemin/k-NN/jni/bin/jni_test)
==30963==    by 0x7BBB8B: void testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) (in /workspace/heemin/k-NN/jni/bin/jni_test)
==30963==    by 0x7A9F5B: testing::Test::Run() [clone .part.0] (in /workspace/heemin/k-NN/jni/bin/jni_test)
==30963==    by 0x7AA52F: testing::TestInfo::Run() (in /workspace/heemin/k-NN/jni/bin/jni_test)
==30963==    by 0x7AB43F: testing::TestSuite::Run() [clone .part.0] (in /workspace/heemin/k-NN/jni/bin/jni_test)
==30963==    by 0x7B2A0F: testing::internal::UnitTestImpl::RunAllTests() (in /workspace/heemin/k-NN/jni/bin/jni_test)
==30963==    by 0x7AA647: testing::UnitTest::Run() (in /workspace/heemin/k-NN/jni/bin/jni_test)
==30963==    by 0x1EDD7F: main (in /workspace/heemin/k-NN/jni/bin/jni_test)
==30963== 
==30963== 131,960 (520 direct, 131,440 indirect) bytes in 1 blocks are definitely lost in loss record 33 of 33
==30963==    at 0x48657B8: operator new(unsigned long) (in /usr/libexec/valgrind/vgpreload_memcheck-arm64-linux.so)
==30963==    by 0x1F09C7: FaissIsSharedIndexStateRequired_BasicAssertions_Test::TestBody() (in /workspace/heemin/k-NN/jni/bin/jni_test)
==30963==    by 0x7BBB8B: void testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) (in /workspace/heemin/k-NN/jni/bin/jni_test)
==30963==    by 0x7A9F5B: testing::Test::Run() [clone .part.0] (in /workspace/heemin/k-NN/jni/bin/jni_test)
==30963==    by 0x7AA52F: testing::TestInfo::Run() (in /workspace/heemin/k-NN/jni/bin/jni_test)
==30963==    by 0x7AB43F: testing::TestSuite::Run() [clone .part.0] (in /workspace/heemin/k-NN/jni/bin/jni_test)
==30963==    by 0x7B2A0F: testing::internal::UnitTestImpl::RunAllTests() (in /workspace/heemin/k-NN/jni/bin/jni_test)
==30963==    by 0x7AA647: testing::UnitTest::Run() (in /workspace/heemin/k-NN/jni/bin/jni_test)
==30963==    by 0x1EDD7F: main (in /workspace/heemin/k-NN/jni/bin/jni_test)
==30963== 
==30963== LEAK SUMMARY:
==30963==    definitely lost: 1,208 bytes in 5 blocks
==30963==    indirectly lost: 262,940 bytes in 13 blocks
==30963==      possibly lost: 3,312 bytes in 9 blocks
==30963==    still reachable: 127,312 bytes in 14 blocks
==30963==         suppressed: 0 bytes in 0 blocks
==30963== Reachable blocks (those to which a pointer was found) are not shown.
==30963== To see them, rerun with: --leak-check=full --show-leak-kinds=all
==30963== 
==30963== Use --track-origins=yes to see where uninitialised values come from
==30963== For lists of detected and suppressed errors, rerun with: -s
==30963== ERROR SUMMARY: 109 errors from 9 contexts (suppressed: 0 from 0)

Signed-off-by: Heemin Kim <heemin@amazon.com>
@heemin32 heemin32 merged commit 4befe2b into opensearch-project:feature/binary-format Jun 28, 2024
48 of 54 checks passed
@heemin32 heemin32 deleted the leak branch June 28, 2024 02:32
heemin32 added a commit that referenced this pull request Jul 2, 2024
Signed-off-by: Heemin Kim <heemin@amazon.com>
heemin32 added a commit that referenced this pull request Jul 3, 2024
Signed-off-by: Heemin Kim <heemin@amazon.com>
heemin32 added a commit that referenced this pull request Jul 3, 2024
Signed-off-by: Heemin Kim <heemin@amazon.com>
heemin32 added a commit that referenced this pull request Jul 3, 2024
Signed-off-by: Heemin Kim <heemin@amazon.com>
heemin32 added a commit that referenced this pull request Jul 3, 2024
Signed-off-by: Heemin Kim <heemin@amazon.com>
heemin32 added a commit that referenced this pull request Jul 3, 2024
Signed-off-by: Heemin Kim <heemin@amazon.com>
heemin32 added a commit that referenced this pull request Jul 3, 2024
Signed-off-by: Heemin Kim <heemin@amazon.com>
heemin32 added a commit that referenced this pull request Jul 3, 2024
* Add jni interface to use a binary hnsw index with faiss (#1747)

Signed-off-by: Heemin Kim <heemin@amazon.com>

* Fix memory leak on test code (#1776)

Signed-off-by: Heemin Kim <heemin@amazon.com>

---------

Signed-off-by: Heemin Kim <heemin@amazon.com>
luyuncheng pushed a commit to luyuncheng/k-NN-1 that referenced this pull request Jul 7, 2024
…roject#1778)

* Add jni interface to use a binary hnsw index with faiss (opensearch-project#1747)

Signed-off-by: Heemin Kim <heemin@amazon.com>

* Fix memory leak on test code (opensearch-project#1776)

Signed-off-by: Heemin Kim <heemin@amazon.com>

---------

Signed-off-by: Heemin Kim <heemin@amazon.com>
luyuncheng pushed a commit to luyuncheng/k-NN-1 that referenced this pull request Jul 7, 2024
…roject#1778)

* Add jni interface to use a binary hnsw index with faiss (opensearch-project#1747)

Signed-off-by: Heemin Kim <heemin@amazon.com>

* Fix memory leak on test code (opensearch-project#1776)

Signed-off-by: Heemin Kim <heemin@amazon.com>

---------

Signed-off-by: Heemin Kim <heemin@amazon.com>
heemin32 added a commit to heemin32/k-NN that referenced this pull request Jul 11, 2024
…roject#1778)

* Add jni interface to use a binary hnsw index with faiss (opensearch-project#1747)

Signed-off-by: Heemin Kim <heemin@amazon.com>

* Fix memory leak on test code (opensearch-project#1776)

Signed-off-by: Heemin Kim <heemin@amazon.com>

---------

Signed-off-by: Heemin Kim <heemin@amazon.com>
heemin32 added a commit that referenced this pull request Jul 11, 2024
* Add jni interface to use a binary hnsw index with faiss (#1747)



* Fix memory leak on test code (#1776)



---------

Signed-off-by: Heemin Kim <heemin@amazon.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants