Added support to query local arrays #3824

guj · 2023-09-23T01:12:02Z

removed unused code sections
added support for Local Arrays
added test for local arrays in python
result can be returned as vector of start/count (global array only), or vector of block ids

2) added support for Local Arrays 3) added test for local arrays in python 4) result can be returned as vector of start/count (global array only), or vector of block ids

…ks are touched

… SST

anagainaru

The changes look good at a first glance, I will try running it later today and see how it works with my code. My concern is with the testing code, we can discuss tomorrow at the meeting.

examples/query/test.cpp

anagainaru · 2023-09-26T14:03:19Z

testing/adios2/performance/query/TestBPQuery.cpp

-    file << "         <range  compare=\"GT\" value=\"6.6\"/>" << std::endl;
+    file << "         <range  compare=\"GT\" value=\"100.6\"/>" << std::endl;
    file << "         <range  compare=\"LT\" value=\"-0.17\"/>" << std::endl;
    file << "         <op value=\"AND\">" << std::endl;
-    file << "            <range  compare=\"LT\" value=\"2.9\"/>" << std::endl;
+    file << "            <range  compare=\"LT\" value=\"11.9\"/>" << std::endl;
    file << "            <range  compare=\"GT\" value=\"2.8\"/>" << std::endl;


I would not use the heat transfer data for the Testing files. In the examples you could use whatever you want, but when testing a function I would use some data that is easy to check correctness for. It is not clear to me why the correct answer for BP4 is {2,1,1}.

@guj , can you address @anagainaru 's concerns?

Sure, I didn't see them

This test does not use heat transfer data. Rather it first writes a file, and then query it.

Add Memory Selection to SST, and prototype a mechanism for running st…

…ios#3834) * Tweak Remote class and test multi-threaded file remote access * rework * Bad clang-format * New formulation

1) Make all examples individually compilable 2) Make examples' executable names be consistent 3) Move certain examples in the right folders 4) Rename certain examples for clarity 5) Make all examples installable 6) Fix Python formatting using black 7) Add licence documentation wherever it's missing

1) Make all examples individually compilable 2) Make examples' executable names be consistent 3) Rename certain examples for clarity 4) Make all examples installable 5) Fix Python formatting using black 6) Add licence documentation wherever it's missing

The CI still enables both.

* Fix usage of DESTDIR for absolute paths * Update cmake/install/post/generate-adios2-config.sh.in --------- Co-authored-by: Tilman Dannert <dannert@rzg.mpg.de>

…amples Organize and add examples

Avoid that for older C-Blosc2 releases, the search logic errors out after the config module was not found.

) Co-authored-by: Greg Eisenhauer <eisen@cc.gatech.edu>

Update examples

* Fix hip example compilation error * Update examples/hello/bpWriteReadHip/CMakeLists.txt --------- Co-authored-by: Vicente Bolea <vicente.bolea@gmail.com>

eisenhauer · 2023-10-17T19:59:38Z

testing/adios2/performance/query/TestBPQuery.cpp

@@ -99,7 +99,7 @@ void BPQueryTest::QueryIntVar(const std::string &fname, adios2::ADIOS &adios,

    std::vector<size_t> rr;
    if (engineName.compare("BP4") == 0)


OK, I'll bite. It looks like the correct output depends upon the engine that you use. Why?

2) added support for Local Arrays 3) added test for local arrays in python 4) result can be returned as vector of start/count (global array only), or vector of block ids

…ks are touched

…queryLocalArray * 'queryLocalArray' of https://github.com/guj/ADIOS2: Update test.cpp modified test. it failed because I consolidate block when all subblocks are touched removed extra ; older clang-format clang-format v16 1) removed unused code sections 2) added support for Local Arrays 3) added test for local arrays in python 4) result can be returned as vector of start/count (global array only), or vector of block ids

guj · 2023-10-23T22:00:44Z

Too many commits after rebase.
Start a new pull request instead: #3868

1) removed unused code sections

f332881

2) added support for Local Arrays 3) added test for local arrays in python 4) result can be returned as vector of start/count (global array only), or vector of block ids

guj requested a review from vicentebolea as a code owner September 23, 2023 01:12

guj and others added 5 commits September 23, 2023 10:02

clang-format v16

1e96b21

older clang-format

7066902

removed extra ;

79528bd

modified test. it failed because I consolidate block when all subbloc…

2149fe7

…ks are touched

Import tests from bp to staging common, implement memory selection in…

2884f71

… SST

guj requested review from pnorbert and anagainaru and removed request for vicentebolea September 25, 2023 16:29

anagainaru reviewed Sep 26, 2023

View reviewed changes

eisenhauer and others added 19 commits September 26, 2023 16:22

Try always using the MPI version

51f7b6f

Try always using the MPI version

5bde490

Merge pull request ornladios#3823 from eisenhauer/SstMemSel

c503940

Add Memory Selection to SST, and prototype a mechanism for running st…

Add prototype testing of remote functionality (ornladios#3830)

700839d

Tweak Remote class and test multi-threaded file remote access (ornlad…

107164d

…ios#3834) * Tweak Remote class and test multi-threaded file remote access * rework * Bad clang-format * New formulation

MPI: add timeout for conf test for MPI_DP (ornladios#3848)

ac07185

Update test.cpp

8707f2b

cmake: update minimum cmake to 3.12 (ornladios#3849)

d0fcf10

MPI_DP: do not call MPI_Init (ornladios#3847)

975e8de

Fix formatting issue in DetectOptions.cmake

fdee65d

Remove testing based on ADIOS2-examples

b881acc

Disable BUILD_TESTING and ADIOS2_BUILD_EXAMPLES by default

bf63c65

The CI still enables both.

Add/Improve the ReadMe.md files in examples directory

2425033

ci: enable tau check

326bb54

cmake: include ctest before detectoptions

2324daf

Fix destdir install test (ornladios#3850)

861ccb7

* Fix usage of DESTDIR for absolute paths * Update cmake/install/post/generate-adios2-config.sh.in --------- Co-authored-by: Tilman Dannert <dannert@rzg.mpg.de>

Merge pull request ornladios#3820 from spyridon97/organize-and-add-ex…

a43d013

…amples Organize and add examples

ax3l and others added 12 commits October 19, 2023 19:41

Blosc2 USE ON: Fix Module Fallback (ornladios#3774)

6f33d87

Avoid that for older C-Blosc2 releases, the search logic errors out after the config module was not found.

BP5Serializer: handle local variables that use operators (ornladios#3859

2f581ec

) Co-authored-by: Greg Eisenhauer <eisen@cc.gatech.edu>

Examples: Add BeginStep/EndStep wherever it was missing

3662939

inlineMWE example: Close files at the end

e13cc18

Examples: Use BPFile instead of BP3/4/5 for future-proof

e98f442

Update the bpWriterReadHip example's cmake to run on crusher

5315ead

Merge pull request ornladios#3856 from spyridon97/update-examples

2d4af78

Update examples

bug fix: syntax error in json output (ornladios#3857)

2426600

Remove Slack as a contact mechanism (ornladios#3866)

347d356

ascent,ci: remove unshallow flag

afdef44

Server Improvements (ornladios#3862)

432c6ab

Fix hip example compilation error (ornladios#3865)

e53a1bf

* Fix hip example compilation error * Update examples/hello/bpWriteReadHip/CMakeLists.txt --------- Co-authored-by: Vicente Bolea <vicente.bolea@gmail.com>

eisenhauer previously approved these changes Oct 23, 2023

View reviewed changes

guj added 8 commits October 23, 2023 10:43

add changes to the query test file (someone moved to a new dir)

de0ad6e

1) removed unused code sections

7322c8d

2) added support for Local Arrays 3) added test for local arrays in python 4) result can be returned as vector of start/count (global array only), or vector of block ids

clang-format v16

7402c20

older clang-format

04a3dbc

removed extra ;

75c34a1

modified test. it failed because I consolidate block when all subbloc…

7c21614

…ks are touched

touched

a5866d2

guj dismissed eisenhauer’s stale review via a88271b October 23, 2023 17:51

guj requested a review from caitlinross as a code owner October 23, 2023 17:51

guj removed the request for review from caitlinross October 23, 2023 19:10

guj closed this Oct 23, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Added support to query local arrays #3824

Added support to query local arrays #3824

guj commented Sep 23, 2023

anagainaru left a comment

anagainaru Sep 26, 2023

eisenhauer Oct 17, 2023

guj Oct 17, 2023

guj Oct 17, 2023

eisenhauer Oct 17, 2023

guj commented Oct 23, 2023

		@@ -99,7 +99,7 @@ void BPQueryTest::QueryIntVar(const std::string &fname, adios2::ADIOS &adios,

		std::vector<size_t> rr;
		if (engineName.compare("BP4") == 0)

Added support to query local arrays #3824

Added support to query local arrays #3824

Conversation

guj commented Sep 23, 2023

anagainaru left a comment

Choose a reason for hiding this comment

anagainaru Sep 26, 2023

Choose a reason for hiding this comment

eisenhauer Oct 17, 2023

Choose a reason for hiding this comment

guj Oct 17, 2023

Choose a reason for hiding this comment

guj Oct 17, 2023

Choose a reason for hiding this comment

eisenhauer Oct 17, 2023

Choose a reason for hiding this comment

guj commented Oct 23, 2023