[SYCL] Add Experimental Range Rounding #12690

hdelan · 2024-02-12T18:07:23Z

This commit adds new clang command line option -fsycl-exp-range-rounding. Experimental range rounding maps all 1, 2 or 3 dim range kernels to range rounded kernels which can be rounded in all dims.

SYCL_PARALLEL_FOR_RANGE_ROUNDING_PARAMS=x:y:z can be used for exp range rounding, where the y param will be a factor of the rounded range in all dims

hdelan · 2024-02-12T18:09:15Z

@intel/dpcpp-clang-driver-reviewers I am not sure about compiler flags. Instead of introducing the new -fsycl-force-range-rounding, should I instead make a new flag -fsycl-range-rounding=[force/disable] which would deprecate the existing -fsycl-disable-range-rounding?

sycl/include/sycl/handler.hpp

mdtoguchi · 2024-02-12T18:33:24Z

@intel/dpcpp-clang-driver-reviewers I am not sure about compiler flags. Instead of introducing the new -fsycl-force-range-rounding, should I instead make a new flag -fsycl-range-rounding=[force/disable] which would deprecate the existing -fsycl-disable-range-rounding?

I like the idea of introducing a new -fsycl-range-rounding=<arg> option. It alleviates any potential confusion when multiple range-rounding options are used on the command line.

aelovikov-intel · 2024-02-12T18:35:49Z

+1 to -fsycl-range-rounding=[disable|on|force]. I'd also split the experimental 1D rounding into a separate PR.

hdelan · 2024-02-14T17:32:01Z

@mdtoguchi @aelovikov-intel changes for the new -fsycl-range-rounding flag are here. I will keep this PR for the experimental range rounding

rafbiels · 2024-03-22T12:34:33Z

I tested this PR with https://gist.github.com/rafbiels/2b584584cfd6412e6b255adab4c264d6 on NVIDIA sm_86 GPU. Here's the outcomes:

   Release     Time [ms]     CUDA gridXYZ     CUDA blockXYZ
-----------------------------------------------------------
  2023.2.1            63     {2, 1888, 1}       {394, 1, 1}
  2024.0.2           223      {788, 3, 1}       {1, 768, 1}
     test1            66     {197, 16, 1}       {4, 118, 1}
     test2            61     {25, 118, 1}       {32, 16, 1}

test1 = hdelan:1d-range-rounding with default flags
test2 = hdelan:1d-range-rounding with -fsycl-exp-range-rounding

The improvement between 2024.0.2 and test1 is thanks to #12663 but we can see that this PR (test2) improves the local range assignment further and the performance surpasses the earlier naive implementation from 2023.2.1.

hdelan · 2024-03-22T16:02:42Z

Ping @againull @intel/llvm-reviewers-runtime @intel/dpcpp-cfe-reviewers @intel/dpcpp-clang-driver-reviewers @intel/unified-runtime-reviewers

aelovikov-intel · 2024-03-22T16:10:36Z

I tested this PR with https://gist.github.com/rafbiels/2b584584cfd6412e6b255adab4c264d6 on NVIDIA sm_86 GPU. Here's the outcomes:

Can we add it into test-e2e/PerformanceTests ?

clang/lib/Driver/ToolChains/Clang.cpp

elizabethandrews

FE changes LGTM

hdelan · 2024-03-22T16:44:50Z

The perf tests stuff is in a very early stage, we're basically finding it out as we go. Currently it's just the logs captured for the tests there even in case of a PASS, the rest is left to humans to interpret.

OK sure thanks

aarongreig

approving to satisfy the requirement but I don't think UR reviewers are responsible for any of the code changed

hdelan · 2024-03-22T16:45:00Z

Actually I just realized there are no FE tests. Please add a test for the predefines.

Test added, thanks.

mdtoguchi

OK for Driver

smanna12

FE changes LGTM

clang/include/clang/Basic/LangOptions.def

This commit adds new clang command line option -fsycl-exp-range-rounding. Experimental range rounding maps all 1, 2 or 3 dim range kernels to a 1d range kernel. This can give performance improvements when inner dimensions are oddly shaped.

In order to get better performance it is beneficial to preserve the dimensionality of the range. This new experimental range rounding does rounding in each dimension. The runtime can take suggestions on the size of the workgroup in all directions using SYCL_PARALLEL_FOR_RANGE_ROUNDING_PARAMS=x:y:z. In the case where -fsycl-exp-range-rounding is used, the middle param is the workgroup size in all dimensions, so for a 1d range kernel, the global range will divide {y}. In 2d the global range will be some number of workgroups of size {y, y}. The same for 3d with {y, y, y}.

Exp range rounding can also be used with the env var: SYCL_PARALLEL_FOR_RANGE_ROUNDING_PARAMS.

Adds the test case where -fsycl-range-rounding=force is used together with -fsycl-exp-range-rounding. This ensures that flags do not interfere with eachother.

Add test that compares the performance of no range rounding, normal range rounding and experimental range rounding.

Clarify that disabling range rounding will override experimental range rounding.

Check for the presence of macro __SYCL_EXP_PARALLEL_FOR_RANGE_ROUNDING__ when using -fsycl-exp-range-rounding.

hdelan · 2024-04-01T10:28:34Z

@intel/llvm-gatekeepers this can be merged now

hdelan requested review from a team as code owners February 12, 2024 18:07

hdelan requested a review from againull February 12, 2024 18:07

hdelan commented Feb 12, 2024

View reviewed changes

sycl/include/sycl/handler.hpp Show resolved Hide resolved

hdelan temporarily deployed to WindowsCILock February 12, 2024 18:23 — with GitHub Actions Inactive

hdelan had a problem deploying to WindowsCILock February 12, 2024 19:31 — with GitHub Actions Failure

hdelan mentioned this pull request Feb 15, 2024

[SYCL] Add force range rounding option and introduce new compiler flag #12715

Merged

hdelan marked this pull request as draft March 20, 2024 15:35

hdelan changed the title ~~[SYCL][Draft] Make some changes for range rounding~~ [SYCL] Add Experimental Range Rounding Mar 20, 2024

hdelan force-pushed the 1d-range-rounding branch from d0adeae to fed0545 Compare March 21, 2024 16:16

hdelan had a problem deploying to WindowsCILock March 21, 2024 16:18 — with GitHub Actions Error

hdelan had a problem deploying to WindowsCILock March 21, 2024 16:59 — with GitHub Actions Error

hdelan force-pushed the 1d-range-rounding branch from 90324d0 to b38a278 Compare March 21, 2024 17:24

hdelan temporarily deployed to WindowsCILock March 21, 2024 17:53 — with GitHub Actions Inactive

hdelan temporarily deployed to WindowsCILock March 21, 2024 19:26 — with GitHub Actions Inactive

intel deleted a comment from github-actions bot Mar 22, 2024

hdelan temporarily deployed to WindowsCILock March 22, 2024 12:47 — with GitHub Actions Inactive

hdelan temporarily deployed to WindowsCILock March 22, 2024 14:07 — with GitHub Actions Inactive

hdelan marked this pull request as ready for review March 22, 2024 16:01

mdtoguchi reviewed Mar 22, 2024

View reviewed changes

clang/lib/Driver/ToolChains/Clang.cpp Show resolved Hide resolved

elizabethandrews approved these changes Mar 22, 2024

View reviewed changes

aarongreig approved these changes Mar 22, 2024

View reviewed changes

mdtoguchi approved these changes Mar 22, 2024

View reviewed changes

smanna12 approved these changes Mar 22, 2024

View reviewed changes

hdelan temporarily deployed to WindowsCILock March 22, 2024 17:39 — with GitHub Actions Inactive

hdelan temporarily deployed to WindowsCILock March 22, 2024 19:54 — with GitHub Actions Inactive

srividya-sundaram reviewed Mar 22, 2024

View reviewed changes

clang/include/clang/Basic/LangOptions.def Show resolved Hide resolved

hdelan temporarily deployed to WindowsCILock March 23, 2024 16:48 — with GitHub Actions Inactive

hdelan temporarily deployed to WindowsCILock March 23, 2024 17:40 — with GitHub Actions Inactive

againull approved these changes Mar 25, 2024

View reviewed changes

hdelan force-pushed the 1d-range-rounding branch from eac2d32 to 856b61f Compare March 28, 2024 09:49

hdelan closed this Mar 28, 2024

hdelan reopened this Mar 28, 2024

hdelan force-pushed the 1d-range-rounding branch from 856b61f to 1965020 Compare March 28, 2024 14:31

hdelan added 9 commits March 29, 2024 09:57

Make some changes for range rounding

2515d72

This commit adds new clang command line option -fsycl-exp-range-rounding. Experimental range rounding maps all 1, 2 or 3 dim range kernels to a 1d range kernel. This can give performance improvements when inner dimensions are oddly shaped.

Add documentation for exp range rounding

9649797

Exp range rounding can also be used with the env var: SYCL_PARALLEL_FOR_RANGE_ROUNDING_PARAMS.

Add combined range rounding flags to test

e324b36

Adds the test case where -fsycl-range-rounding=force is used together with -fsycl-exp-range-rounding. This ensures that flags do not interfere with eachother.

Fix lint

6f850d0

Add test to PerformanceTests

bf66eb5

Add test that compares the performance of no range rounding, normal range rounding and experimental range rounding.

Update docs

7e4082b

Clarify that disabling range rounding will override experimental range rounding.

Remove a.out

b0d5765

Add clang test

17de821

Check for the presence of macro __SYCL_EXP_PARALLEL_FOR_RANGE_ROUNDING__ when using -fsycl-exp-range-rounding.

hdelan force-pushed the 1d-range-rounding branch from 1965020 to 17de821 Compare March 29, 2024 09:57

hdelan temporarily deployed to WindowsCILock March 29, 2024 09:59 — with GitHub Actions Inactive

hdelan temporarily deployed to WindowsCILock March 29, 2024 11:26 — with GitHub Actions Inactive

martygrant merged commit 319cf0e into intel:sycl Apr 1, 2024
16 checks passed

hdelan mentioned this pull request Apr 10, 2024

Poor performance for 2d/3d range kernels involving primes on CUDA PI #8018

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[SYCL] Add Experimental Range Rounding #12690

[SYCL] Add Experimental Range Rounding #12690

hdelan commented Feb 12, 2024 •

edited

Loading

hdelan commented Feb 12, 2024

mdtoguchi commented Feb 12, 2024

aelovikov-intel commented Feb 12, 2024

hdelan commented Feb 14, 2024

rafbiels commented Mar 22, 2024

hdelan commented Mar 22, 2024

aelovikov-intel commented Mar 22, 2024

elizabethandrews left a comment

hdelan commented Mar 22, 2024

aarongreig left a comment

hdelan commented Mar 22, 2024

mdtoguchi left a comment

smanna12 left a comment

hdelan commented Apr 1, 2024

[SYCL] Add Experimental Range Rounding #12690

[SYCL] Add Experimental Range Rounding #12690

Conversation

hdelan commented Feb 12, 2024 • edited Loading

hdelan commented Feb 12, 2024

mdtoguchi commented Feb 12, 2024

aelovikov-intel commented Feb 12, 2024

hdelan commented Feb 14, 2024

rafbiels commented Mar 22, 2024

hdelan commented Mar 22, 2024

aelovikov-intel commented Mar 22, 2024

elizabethandrews left a comment

Choose a reason for hiding this comment

hdelan commented Mar 22, 2024

aarongreig left a comment

Choose a reason for hiding this comment

hdelan commented Mar 22, 2024

mdtoguchi left a comment

Choose a reason for hiding this comment

smanna12 left a comment

Choose a reason for hiding this comment

hdelan commented Apr 1, 2024

hdelan commented Feb 12, 2024 •

edited

Loading