Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

SpMV: adding support for rocSPARSE TPL #1221

Merged
merged 2 commits into from
Dec 20, 2021
Merged

SpMV: adding support for rocSPARSE TPL #1221

merged 2 commits into from
Dec 20, 2021

Conversation

lucbv
Copy link
Contributor

@lucbv lucbv commented Dec 8, 2021

rocSPARSE is declared as a TPL for HIP backend and hooks are in place to call the TPL when the library supports it (mode and types restrictions apply). This should provide reasonable performance to the users when the TPL is available while we work on further tuning the native launch parameters of SpMV.
The unit-tests are passing fine, the performance test is now too old and outdated to be updated easily. Further work will be needed to get that done.

This was tested on spock with all unit-test passing.

@lucbv
SpMV: adding support for rocSPARSE TPL
256144e 
rocSPARSE is declared as a TPL for HIP backend
and hooks are in place to call the TPL when the
library supports it (mode and types restrictions
apply). The unit-tests are passing fine, the
performance test is now too old and outdated
to be updated easily. Further work will be needed
to get that done.
@lucbv lucbv self-assigned this Dec 8, 2021
@kokkos-devops-admin
Copy link

Status Flag 'Pre-Test Inspection' - Auto Inspected - Inspection Is Not Necessary for this Pull Request.

@kokkos-devops-admin
Copy link

Status Flag 'Pull Request AutoTester' - Testing Jenkins Projects:

Pull Request Auto Testing STARTING (click to expand)

Build Information

Test Name: KokkosKernels_PullRequest_Tpls_CUDA9_GCC720_Light_Tpls_GCC720_GCC740

  • Build Num: 18
  • Status: STARTED

Jenkins Parameters

Parameter Name Value
KOKKOSKERNELS_SOURCE_BRANCH SPMV_rocsparse
KOKKOSKERNELS_SOURCE_REPO https://github.com/lucbv/kokkos-kernels
KOKKOSKERNELS_SOURCE_SHA 35dd6c4
KOKKOSKERNELS_TARGET_BRANCH develop
KOKKOSKERNELS_TARGET_REPO https://github.com/kokkos/kokkos-kernels
KOKKOSKERNELS_TARGET_SHA 7413f2a
PR_LABELS enhancement;feature request
PULLREQUESTNUM 1221
TEST_REPO_ALIAS KOKKOSKERNELS

Build Information

Test Name: KokkosKernels_PullRequest_Tpls_CUDA10_Tpls_CUDA10_LayoutRight

  • Build Num: 18
  • Status: STARTED

Jenkins Parameters

Parameter Name Value
KOKKOSKERNELS_SOURCE_BRANCH SPMV_rocsparse
KOKKOSKERNELS_SOURCE_REPO https://github.com/lucbv/kokkos-kernels
KOKKOSKERNELS_SOURCE_SHA 35dd6c4
KOKKOSKERNELS_TARGET_BRANCH develop
KOKKOSKERNELS_TARGET_REPO https://github.com/kokkos/kokkos-kernels
KOKKOSKERNELS_TARGET_SHA 7413f2a
PR_LABELS enhancement;feature request
PULLREQUESTNUM 1221
TEST_REPO_ALIAS KOKKOSKERNELS

Build Information

Test Name: KokkosKernels_PullRequest_GCC720

  • Build Num: 676
  • Status: STARTED

Jenkins Parameters

Parameter Name Value
KOKKOSKERNELS_SOURCE_BRANCH SPMV_rocsparse
KOKKOSKERNELS_SOURCE_REPO https://github.com/lucbv/kokkos-kernels
KOKKOSKERNELS_SOURCE_SHA 35dd6c4
KOKKOSKERNELS_TARGET_BRANCH develop
KOKKOSKERNELS_TARGET_REPO https://github.com/kokkos/kokkos-kernels
KOKKOSKERNELS_TARGET_SHA 7413f2a
PR_LABELS enhancement;feature request
PULLREQUESTNUM 1221
TEST_REPO_ALIAS KOKKOSKERNELS

Build Information

Test Name: KokkosKernels_PullRequest_GCC720_Light_LayoutRight

  • Build Num: 323
  • Status: STARTED

Jenkins Parameters

Parameter Name Value
KOKKOSKERNELS_SOURCE_BRANCH SPMV_rocsparse
KOKKOSKERNELS_SOURCE_REPO https://github.com/lucbv/kokkos-kernels
KOKKOSKERNELS_SOURCE_SHA 35dd6c4
KOKKOSKERNELS_TARGET_BRANCH develop
KOKKOSKERNELS_TARGET_REPO https://github.com/kokkos/kokkos-kernels
KOKKOSKERNELS_TARGET_SHA 7413f2a
PR_LABELS enhancement;feature request
PULLREQUESTNUM 1221
TEST_REPO_ALIAS KOKKOSKERNELS

Build Information

Test Name: KokkosKernels_PullRequest_Tpls_GCC720

  • Build Num: 667
  • Status: STARTED

Jenkins Parameters

Parameter Name Value
KOKKOSKERNELS_SOURCE_BRANCH SPMV_rocsparse
KOKKOSKERNELS_SOURCE_REPO https://github.com/lucbv/kokkos-kernels
KOKKOSKERNELS_SOURCE_SHA 35dd6c4
KOKKOSKERNELS_TARGET_BRANCH develop
KOKKOSKERNELS_TARGET_REPO https://github.com/kokkos/kokkos-kernels
KOKKOSKERNELS_TARGET_SHA 7413f2a
PR_LABELS enhancement;feature request
PULLREQUESTNUM 1221
TEST_REPO_ALIAS KOKKOSKERNELS

Build Information

Test Name: KokkosKernels_PullRequest_Tpls_INTEL18

  • Build Num: 654
  • Status: STARTED

Jenkins Parameters

Parameter Name Value
KOKKOSKERNELS_SOURCE_BRANCH SPMV_rocsparse
KOKKOSKERNELS_SOURCE_REPO https://github.com/lucbv/kokkos-kernels
KOKKOSKERNELS_SOURCE_SHA 35dd6c4
KOKKOSKERNELS_TARGET_BRANCH develop
KOKKOSKERNELS_TARGET_REPO https://github.com/kokkos/kokkos-kernels
KOKKOSKERNELS_TARGET_SHA 7413f2a
PR_LABELS enhancement;feature request
PULLREQUESTNUM 1221
TEST_REPO_ALIAS KOKKOSKERNELS

Build Information

Test Name: KokkosKernels_PullRequest_CLANG1001

  • Build Num: 60
  • Status: STARTED

Jenkins Parameters

Parameter Name Value
KOKKOSKERNELS_SOURCE_BRANCH SPMV_rocsparse
KOKKOSKERNELS_SOURCE_REPO https://github.com/lucbv/kokkos-kernels
KOKKOSKERNELS_SOURCE_SHA 35dd6c4
KOKKOSKERNELS_TARGET_BRANCH develop
KOKKOSKERNELS_TARGET_REPO https://github.com/kokkos/kokkos-kernels
KOKKOSKERNELS_TARGET_SHA 7413f2a
PR_LABELS enhancement;feature request
PULLREQUESTNUM 1221
TEST_REPO_ALIAS KOKKOSKERNELS

Using Repos:

Repo: KOKKOSKERNELS (lucbv/kokkos-kernels)
  • Branch: SPMV_rocsparse
  • SHA: 35dd6c4
  • Mode: TEST_REPO

Pull Request Author: lucbv

@kokkos-devops-admin
Copy link

Status Flag 'Pull Request AutoTester' - Jenkins Testing: all Jobs PASSED

Pull Request Auto Testing has PASSED (click to expand)

Build Information

Test Name: KokkosKernels_PullRequest_Tpls_CUDA9_GCC720_Light_Tpls_GCC720_GCC740

  • Build Num: 18
  • Status: PASSED

Jenkins Parameters

Parameter Name Value
KOKKOSKERNELS_SOURCE_BRANCH SPMV_rocsparse
KOKKOSKERNELS_SOURCE_REPO https://github.com/lucbv/kokkos-kernels
KOKKOSKERNELS_SOURCE_SHA 35dd6c4
KOKKOSKERNELS_TARGET_BRANCH develop
KOKKOSKERNELS_TARGET_REPO https://github.com/kokkos/kokkos-kernels
KOKKOSKERNELS_TARGET_SHA 7413f2a
PR_LABELS enhancement;feature request
PULLREQUESTNUM 1221
TEST_REPO_ALIAS KOKKOSKERNELS

Build Information

Test Name: KokkosKernels_PullRequest_Tpls_CUDA10_Tpls_CUDA10_LayoutRight

  • Build Num: 18
  • Status: PASSED

Jenkins Parameters

Parameter Name Value
KOKKOSKERNELS_SOURCE_BRANCH SPMV_rocsparse
KOKKOSKERNELS_SOURCE_REPO https://github.com/lucbv/kokkos-kernels
KOKKOSKERNELS_SOURCE_SHA 35dd6c4
KOKKOSKERNELS_TARGET_BRANCH develop
KOKKOSKERNELS_TARGET_REPO https://github.com/kokkos/kokkos-kernels
KOKKOSKERNELS_TARGET_SHA 7413f2a
PR_LABELS enhancement;feature request
PULLREQUESTNUM 1221
TEST_REPO_ALIAS KOKKOSKERNELS

Build Information

Test Name: KokkosKernels_PullRequest_GCC720

  • Build Num: 676
  • Status: PASSED

Jenkins Parameters

Parameter Name Value
KOKKOSKERNELS_SOURCE_BRANCH SPMV_rocsparse
KOKKOSKERNELS_SOURCE_REPO https://github.com/lucbv/kokkos-kernels
KOKKOSKERNELS_SOURCE_SHA 35dd6c4
KOKKOSKERNELS_TARGET_BRANCH develop
KOKKOSKERNELS_TARGET_REPO https://github.com/kokkos/kokkos-kernels
KOKKOSKERNELS_TARGET_SHA 7413f2a
PR_LABELS enhancement;feature request
PULLREQUESTNUM 1221
TEST_REPO_ALIAS KOKKOSKERNELS

Build Information

Test Name: KokkosKernels_PullRequest_GCC720_Light_LayoutRight

  • Build Num: 323
  • Status: PASSED

Jenkins Parameters

Parameter Name Value
KOKKOSKERNELS_SOURCE_BRANCH SPMV_rocsparse
KOKKOSKERNELS_SOURCE_REPO https://github.com/lucbv/kokkos-kernels
KOKKOSKERNELS_SOURCE_SHA 35dd6c4
KOKKOSKERNELS_TARGET_BRANCH develop
KOKKOSKERNELS_TARGET_REPO https://github.com/kokkos/kokkos-kernels
KOKKOSKERNELS_TARGET_SHA 7413f2a
PR_LABELS enhancement;feature request
PULLREQUESTNUM 1221
TEST_REPO_ALIAS KOKKOSKERNELS

Build Information

Test Name: KokkosKernels_PullRequest_Tpls_GCC720

  • Build Num: 667
  • Status: PASSED

Jenkins Parameters

Parameter Name Value
KOKKOSKERNELS_SOURCE_BRANCH SPMV_rocsparse
KOKKOSKERNELS_SOURCE_REPO https://github.com/lucbv/kokkos-kernels
KOKKOSKERNELS_SOURCE_SHA 35dd6c4
KOKKOSKERNELS_TARGET_BRANCH develop
KOKKOSKERNELS_TARGET_REPO https://github.com/kokkos/kokkos-kernels
KOKKOSKERNELS_TARGET_SHA 7413f2a
PR_LABELS enhancement;feature request
PULLREQUESTNUM 1221
TEST_REPO_ALIAS KOKKOSKERNELS

Build Information

Test Name: KokkosKernels_PullRequest_Tpls_INTEL18

  • Build Num: 654
  • Status: PASSED

Jenkins Parameters

Parameter Name Value
KOKKOSKERNELS_SOURCE_BRANCH SPMV_rocsparse
KOKKOSKERNELS_SOURCE_REPO https://github.com/lucbv/kokkos-kernels
KOKKOSKERNELS_SOURCE_SHA 35dd6c4
KOKKOSKERNELS_TARGET_BRANCH develop
KOKKOSKERNELS_TARGET_REPO https://github.com/kokkos/kokkos-kernels
KOKKOSKERNELS_TARGET_SHA 7413f2a
PR_LABELS enhancement;feature request
PULLREQUESTNUM 1221
TEST_REPO_ALIAS KOKKOSKERNELS

Build Information

Test Name: KokkosKernels_PullRequest_CLANG1001

  • Build Num: 60
  • Status: PASSED

Jenkins Parameters

Parameter Name Value
KOKKOSKERNELS_SOURCE_BRANCH SPMV_rocsparse
KOKKOSKERNELS_SOURCE_REPO https://github.com/lucbv/kokkos-kernels
KOKKOSKERNELS_SOURCE_SHA 35dd6c4
KOKKOSKERNELS_TARGET_BRANCH develop
KOKKOSKERNELS_TARGET_REPO https://github.com/kokkos/kokkos-kernels
KOKKOSKERNELS_TARGET_SHA 7413f2a
PR_LABELS enhancement;feature request
PULLREQUESTNUM 1221
TEST_REPO_ALIAS KOKKOSKERNELS

@kokkos-devops-admin
Copy link

Status Flag 'Pre-Merge Inspection' - - This Pull Request Requires Inspection... The code must be inspected by a member of the Team before Testing/Merging
NO REVIEWS HAVE BEEN PERFORMED ON THIS PULL REQUEST!

@kokkos-devops-admin
Copy link

All Jobs Finished; status = PASSED, However Inspection must be performed before merge can occur...

@iyamazaki
Copy link
Contributor

Thank you @lucbv. It looks good to me, but it will be nice if @brian-kelley can also look. I have a couple of questions about rocSparse:

  • with rocSparse, offset_type and entry_type can be different?
  • It seems to allocate a buffer for each SpMV? Is that not much overhead?
  • I guess this is just with one vector? Will you have a support for multiple vectors?

@kokkos-devops-admin
Copy link

All Jobs Finished; status = PASSED, However Inspection must be performed before merge can occur...

3 similar comments
@kokkos-devops-admin
Copy link

All Jobs Finished; status = PASSED, However Inspection must be performed before merge can occur...

@kokkos-devops-admin
Copy link

All Jobs Finished; status = PASSED, However Inspection must be performed before merge can occur...

@kokkos-devops-admin
Copy link

All Jobs Finished; status = PASSED, However Inspection must be performed before merge can occur...

@lucbv
Copy link
Contributor Author

lucbv commented Dec 13, 2021

@brian-kelley can you give this a look please?

@kokkos-devops-admin
Copy link

Status Flag 'Pre-Merge Inspection' - SUCCESS: The last commit to this Pull Request has been INSPECTED AND APPROVED by [ brian-kelley ]!

@kokkos-devops-admin
Copy link

Status Flag 'Pull Request AutoTester' - Pull Request MUST BE MERGED MANUALLY BY Project Team - This Repo does not support Automerge

@brian-kelley
Copy link
Contributor

It would be nice eventually to just instantiate for the 'unified' layout (Left or Right) instead of instantiating for both, but other TPLs have the same issue and it's outside the scope of just getting rocsparse working.

@kokkos-devops-admin
Copy link

Status Flag 'Pull Request AutoTester' - Pull Request MUST BE MERGED MANUALLY BY Project Team - This Repo does not support Automerge

@kokkos-devops-admin
Copy link

All Jobs Finished; status = PASSED, However PR is now STALE, and must be retested. Set the AT: RETEST Label to force retest....

5 similar comments
@kokkos-devops-admin
Copy link

All Jobs Finished; status = PASSED, However PR is now STALE, and must be retested. Set the AT: RETEST Label to force retest....

@kokkos-devops-admin
Copy link

All Jobs Finished; status = PASSED, However PR is now STALE, and must be retested. Set the AT: RETEST Label to force retest....

@kokkos-devops-admin
Copy link

All Jobs Finished; status = PASSED, However PR is now STALE, and must be retested. Set the AT: RETEST Label to force retest....

@kokkos-devops-admin
Copy link

All Jobs Finished; status = PASSED, However PR is now STALE, and must be retested. Set the AT: RETEST Label to force retest....

@kokkos-devops-admin
Copy link

All Jobs Finished; status = PASSED, However PR is now STALE, and must be retested. Set the AT: RETEST Label to force retest....

@lucbv lucbv merged commit 6c786cd into kokkos:develop Dec 20, 2021
@lucbv lucbv deleted the SPMV_rocsparse branch December 20, 2021 15:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@brian-kelley brian-kelley brian-kelley approved these changes

@iyamazaki iyamazaki Awaiting requested review from iyamazaki

Assignees

@lucbv lucbv

Labels
AT: STALE enhancement feature request
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@lucbv @kokkos-devops-admin @iyamazaki @brian-kelley