Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Bspgemm cusparse hang #2008

Merged
merged 2 commits into from
Oct 23, 2023
Merged

Bspgemm cusparse hang #2008

merged 2 commits into from
Oct 23, 2023

Conversation

lucbv
Copy link
Contributor

@lucbv lucbv commented Oct 20, 2023

This re-enables the tests for spgemm and bspgemm but turns off the spec_avail struct for the particular versions of cusparse that hang.
Let me know if that sounds reasonable?
I believe that is what @srajama1 prefers to fix the issue.

lucbv added 2 commits October 20, 2023 10:57
@lucbv
Sparse: fix cusparse spgemm hang properly
a3dd32b 
The issue is fixed by disabling the TPL in spec_avail
when a problematic version of CUDA/cuSPARSE is being used.
@lucbv
Sparse: fix logic for bad cursparse spgemm version.
6680347 
Just inverted the logic statement to enable the TPL when it is
known to work correctly.
@lucbv lucbv added bug Cleanup Code maintenance that isn't a bugfix or new feature labels Oct 20, 2023
@lucbv lucbv self-assigned this Oct 20, 2023
@kokkos-devops-admin
Copy link

Status Flag 'Pre-Test Inspection' - Auto Inspected - Inspection is Not Necessary for this Pull Request.

@kokkos-devops-admin
Copy link

Status Flag 'Pull Request AutoTester' - Testing Jenkins Projects:

Pull Request Auto Testing STARTING (click to expand)

Build Information

Test Name: KokkosKernels_PullRequest_CUDA11_CUDA11_LayoutRight

  • Build Num: 995
  • Status: STARTED

Jenkins Parameters

Parameter Name Value
KOKKOSKERNELS_SOURCE_REPO https://github.com/lucbv/kokkos-kernels
KOKKOSKERNELS_SOURCE_SHA 6680347
KOKKOSKERNELS_TARGET_BRANCH develop
KOKKOSKERNELS_TARGET_REPO https://github.com/kokkos/kokkos-kernels
KOKKOSKERNELS_TARGET_SHA c3646ed
PR_LABELS bug;Cleanup
PULLREQUESTNUM 2008
TEST_REPO_ALIAS KOKKOSKERNELS

Build Information

Test Name: KokkosKernels_PullRequest_GCC930_Light_Tpls_GCC930_Tpls_CLANG13CUDA10

  • Build Num: 584
  • Status: STARTED

Jenkins Parameters

Parameter Name Value
KOKKOSKERNELS_SOURCE_REPO https://github.com/lucbv/kokkos-kernels
KOKKOSKERNELS_SOURCE_SHA 6680347
KOKKOSKERNELS_TARGET_BRANCH develop
KOKKOSKERNELS_TARGET_REPO https://github.com/kokkos/kokkos-kernels
KOKKOSKERNELS_TARGET_SHA c3646ed
PR_LABELS bug;Cleanup
PULLREQUESTNUM 2008
TEST_REPO_ALIAS KOKKOSKERNELS

Build Information

Test Name: KokkosKernels_PullRequest_GNU1021

  • Build Num: 267
  • Status: STARTED

Jenkins Parameters

Parameter Name Value
KOKKOSKERNELS_SOURCE_REPO https://github.com/lucbv/kokkos-kernels
KOKKOSKERNELS_SOURCE_SHA 6680347
KOKKOSKERNELS_TARGET_BRANCH develop
KOKKOSKERNELS_TARGET_REPO https://github.com/kokkos/kokkos-kernels
KOKKOSKERNELS_TARGET_SHA c3646ed
PR_LABELS bug;Cleanup
PULLREQUESTNUM 2008
TEST_REPO_ALIAS KOKKOSKERNELS

Build Information

Test Name: KokkosKernels_PullRequest_GNU1021_Light_LayoutRight

  • Build Num: 254
  • Status: STARTED

Jenkins Parameters

Parameter Name Value
KOKKOSKERNELS_SOURCE_REPO https://github.com/lucbv/kokkos-kernels
KOKKOSKERNELS_SOURCE_SHA 6680347
KOKKOSKERNELS_TARGET_BRANCH develop
KOKKOSKERNELS_TARGET_REPO https://github.com/kokkos/kokkos-kernels
KOKKOSKERNELS_TARGET_SHA c3646ed
PR_LABELS bug;Cleanup
PULLREQUESTNUM 2008
TEST_REPO_ALIAS KOKKOSKERNELS

Build Information

Test Name: KokkosKernels_PullRequest_Tpls_GNU1021

  • Build Num: 255
  • Status: STARTED

Jenkins Parameters

Parameter Name Value
KOKKOSKERNELS_SOURCE_REPO https://github.com/lucbv/kokkos-kernels
KOKKOSKERNELS_SOURCE_SHA 6680347
KOKKOSKERNELS_TARGET_BRANCH develop
KOKKOSKERNELS_TARGET_REPO https://github.com/kokkos/kokkos-kernels
KOKKOSKERNELS_TARGET_SHA c3646ed
PR_LABELS bug;Cleanup
PULLREQUESTNUM 2008
TEST_REPO_ALIAS KOKKOSKERNELS

Build Information

Test Name: KokkosKernels_PullRequest_Tpls_INTEL19_solo

  • Build Num: 259
  • Status: STARTED

Jenkins Parameters

Parameter Name Value
KOKKOSKERNELS_SOURCE_REPO https://github.com/lucbv/kokkos-kernels
KOKKOSKERNELS_SOURCE_SHA 6680347
KOKKOSKERNELS_TARGET_BRANCH develop
KOKKOSKERNELS_TARGET_REPO https://github.com/kokkos/kokkos-kernels
KOKKOSKERNELS_TARGET_SHA c3646ed
PR_LABELS bug;Cleanup
PULLREQUESTNUM 2008
TEST_REPO_ALIAS KOKKOSKERNELS

Build Information

Test Name: KokkosKernels_PullRequest_CLANG1001_solo

  • Build Num: 231
  • Status: STARTED

Jenkins Parameters

Parameter Name Value
KOKKOSKERNELS_SOURCE_REPO https://github.com/lucbv/kokkos-kernels
KOKKOSKERNELS_SOURCE_SHA 6680347
KOKKOSKERNELS_TARGET_BRANCH develop
KOKKOSKERNELS_TARGET_REPO https://github.com/kokkos/kokkos-kernels
KOKKOSKERNELS_TARGET_SHA c3646ed
PR_LABELS bug;Cleanup
PULLREQUESTNUM 2008
TEST_REPO_ALIAS KOKKOSKERNELS

Build Information

Test Name: KokkosKernels_PullRequest_A64FX_Tpls_ARMPL2110

  • Build Num: 741
  • Status: STARTED

Jenkins Parameters

Parameter Name Value
KOKKOSKERNELS_SOURCE_REPO https://github.com/lucbv/kokkos-kernels
KOKKOSKERNELS_SOURCE_SHA 6680347
KOKKOSKERNELS_TARGET_BRANCH develop
KOKKOSKERNELS_TARGET_REPO https://github.com/kokkos/kokkos-kernels
KOKKOSKERNELS_TARGET_SHA c3646ed
PR_LABELS bug;Cleanup
PULLREQUESTNUM 2008
TEST_REPO_ALIAS KOKKOSKERNELS

Build Information

Test Name: KokkosKernels_PullRequest_A64FX_GCC1020

  • Build Num: 736
  • Status: STARTED

Jenkins Parameters

Parameter Name Value
KOKKOSKERNELS_SOURCE_REPO https://github.com/lucbv/kokkos-kernels
KOKKOSKERNELS_SOURCE_SHA 6680347
KOKKOSKERNELS_TARGET_BRANCH develop
KOKKOSKERNELS_TARGET_REPO https://github.com/kokkos/kokkos-kernels
KOKKOSKERNELS_TARGET_SHA c3646ed
PR_LABELS bug;Cleanup
PULLREQUESTNUM 2008
TEST_REPO_ALIAS KOKKOSKERNELS

Build Information

Test Name: KokkosKernels_PullRequest_VEGA90A_ROCM560

  • Build Num: 734
  • Status: STARTED

Jenkins Parameters

Parameter Name Value
KOKKOSKERNELS_SOURCE_REPO https://github.com/lucbv/kokkos-kernels
KOKKOSKERNELS_SOURCE_SHA 6680347
KOKKOSKERNELS_TARGET_BRANCH develop
KOKKOSKERNELS_TARGET_REPO https://github.com/kokkos/kokkos-kernels
KOKKOSKERNELS_TARGET_SHA c3646ed
PR_LABELS bug;Cleanup
PULLREQUESTNUM 2008
TEST_REPO_ALIAS KOKKOSKERNELS

Build Information

Test Name: KokkosKernels_PullRequest_VEGA90A_Tpls_ROCM560

  • Build Num: 256
  • Status: STARTED

Jenkins Parameters

Parameter Name Value
KOKKOSKERNELS_SOURCE_REPO https://github.com/lucbv/kokkos-kernels
KOKKOSKERNELS_SOURCE_SHA 6680347
KOKKOSKERNELS_TARGET_BRANCH develop
KOKKOSKERNELS_TARGET_REPO https://github.com/kokkos/kokkos-kernels
KOKKOSKERNELS_TARGET_SHA c3646ed
PR_LABELS bug;Cleanup
PULLREQUESTNUM 2008
TEST_REPO_ALIAS KOKKOSKERNELS

Using Repos:

Repo: KOKKOSKERNELS (lucbv/kokkos-kernels)
  • Branch: bspgemm_cusparse_hang
  • SHA: 6680347
  • Mode: TEST_REPO

Pull Request Author: lucbv

@kokkos-devops-admin
Copy link

Status Flag 'Pull Request AutoTester' - Jenkins Testing: all Jobs PASSED

Pull Request Auto Testing has PASSED (click to expand)

Build Information

Test Name: KokkosKernels_PullRequest_CUDA11_CUDA11_LayoutRight

  • Build Num: 995
  • Status: PASSED

Jenkins Parameters

Parameter Name Value
KOKKOSKERNELS_SOURCE_REPO https://github.com/lucbv/kokkos-kernels
KOKKOSKERNELS_SOURCE_SHA 6680347
KOKKOSKERNELS_TARGET_BRANCH develop
KOKKOSKERNELS_TARGET_REPO https://github.com/kokkos/kokkos-kernels
KOKKOSKERNELS_TARGET_SHA c3646ed
PR_LABELS bug;Cleanup
PULLREQUESTNUM 2008
TEST_REPO_ALIAS KOKKOSKERNELS

Build Information

Test Name: KokkosKernels_PullRequest_GCC930_Light_Tpls_GCC930_Tpls_CLANG13CUDA10

  • Build Num: 584
  • Status: PASSED

Jenkins Parameters

Parameter Name Value
KOKKOSKERNELS_SOURCE_REPO https://github.com/lucbv/kokkos-kernels
KOKKOSKERNELS_SOURCE_SHA 6680347
KOKKOSKERNELS_TARGET_BRANCH develop
KOKKOSKERNELS_TARGET_REPO https://github.com/kokkos/kokkos-kernels
KOKKOSKERNELS_TARGET_SHA c3646ed
PR_LABELS bug;Cleanup
PULLREQUESTNUM 2008
TEST_REPO_ALIAS KOKKOSKERNELS

Build Information

Test Name: KokkosKernels_PullRequest_GNU1021

  • Build Num: 267
  • Status: PASSED

Jenkins Parameters

Parameter Name Value
KOKKOSKERNELS_SOURCE_REPO https://github.com/lucbv/kokkos-kernels
KOKKOSKERNELS_SOURCE_SHA 6680347
KOKKOSKERNELS_TARGET_BRANCH develop
KOKKOSKERNELS_TARGET_REPO https://github.com/kokkos/kokkos-kernels
KOKKOSKERNELS_TARGET_SHA c3646ed
PR_LABELS bug;Cleanup
PULLREQUESTNUM 2008
TEST_REPO_ALIAS KOKKOSKERNELS

Build Information

Test Name: KokkosKernels_PullRequest_GNU1021_Light_LayoutRight

  • Build Num: 254
  • Status: PASSED

Jenkins Parameters

Parameter Name Value
KOKKOSKERNELS_SOURCE_REPO https://github.com/lucbv/kokkos-kernels
KOKKOSKERNELS_SOURCE_SHA 6680347
KOKKOSKERNELS_TARGET_BRANCH develop
KOKKOSKERNELS_TARGET_REPO https://github.com/kokkos/kokkos-kernels
KOKKOSKERNELS_TARGET_SHA c3646ed
PR_LABELS bug;Cleanup
PULLREQUESTNUM 2008
TEST_REPO_ALIAS KOKKOSKERNELS

Build Information

Test Name: KokkosKernels_PullRequest_Tpls_GNU1021

  • Build Num: 255
  • Status: PASSED

Jenkins Parameters

Parameter Name Value
KOKKOSKERNELS_SOURCE_REPO https://github.com/lucbv/kokkos-kernels
KOKKOSKERNELS_SOURCE_SHA 6680347
KOKKOSKERNELS_TARGET_BRANCH develop
KOKKOSKERNELS_TARGET_REPO https://github.com/kokkos/kokkos-kernels
KOKKOSKERNELS_TARGET_SHA c3646ed
PR_LABELS bug;Cleanup
PULLREQUESTNUM 2008
TEST_REPO_ALIAS KOKKOSKERNELS

Build Information

Test Name: KokkosKernels_PullRequest_Tpls_INTEL19_solo

  • Build Num: 259
  • Status: PASSED

Jenkins Parameters

Parameter Name Value
KOKKOSKERNELS_SOURCE_REPO https://github.com/lucbv/kokkos-kernels
KOKKOSKERNELS_SOURCE_SHA 6680347
KOKKOSKERNELS_TARGET_BRANCH develop
KOKKOSKERNELS_TARGET_REPO https://github.com/kokkos/kokkos-kernels
KOKKOSKERNELS_TARGET_SHA c3646ed
PR_LABELS bug;Cleanup
PULLREQUESTNUM 2008
TEST_REPO_ALIAS KOKKOSKERNELS

Build Information

Test Name: KokkosKernels_PullRequest_CLANG1001_solo

  • Build Num: 231
  • Status: PASSED

Jenkins Parameters

Parameter Name Value
KOKKOSKERNELS_SOURCE_REPO https://github.com/lucbv/kokkos-kernels
KOKKOSKERNELS_SOURCE_SHA 6680347
KOKKOSKERNELS_TARGET_BRANCH develop
KOKKOSKERNELS_TARGET_REPO https://github.com/kokkos/kokkos-kernels
KOKKOSKERNELS_TARGET_SHA c3646ed
PR_LABELS bug;Cleanup
PULLREQUESTNUM 2008
TEST_REPO_ALIAS KOKKOSKERNELS

Build Information

Test Name: KokkosKernels_PullRequest_A64FX_Tpls_ARMPL2110

  • Build Num: 741
  • Status: PASSED

Jenkins Parameters

Parameter Name Value
KOKKOSKERNELS_SOURCE_REPO https://github.com/lucbv/kokkos-kernels
KOKKOSKERNELS_SOURCE_SHA 6680347
KOKKOSKERNELS_TARGET_BRANCH develop
KOKKOSKERNELS_TARGET_REPO https://github.com/kokkos/kokkos-kernels
KOKKOSKERNELS_TARGET_SHA c3646ed
PR_LABELS bug;Cleanup
PULLREQUESTNUM 2008
TEST_REPO_ALIAS KOKKOSKERNELS

Build Information

Test Name: KokkosKernels_PullRequest_A64FX_GCC1020

  • Build Num: 736
  • Status: PASSED

Jenkins Parameters

Parameter Name Value
KOKKOSKERNELS_SOURCE_REPO https://github.com/lucbv/kokkos-kernels
KOKKOSKERNELS_SOURCE_SHA 6680347
KOKKOSKERNELS_TARGET_BRANCH develop
KOKKOSKERNELS_TARGET_REPO https://github.com/kokkos/kokkos-kernels
KOKKOSKERNELS_TARGET_SHA c3646ed
PR_LABELS bug;Cleanup
PULLREQUESTNUM 2008
TEST_REPO_ALIAS KOKKOSKERNELS

Build Information

Test Name: KokkosKernels_PullRequest_VEGA90A_ROCM560

  • Build Num: 734
  • Status: PASSED

Jenkins Parameters

Parameter Name Value
KOKKOSKERNELS_SOURCE_REPO https://github.com/lucbv/kokkos-kernels
KOKKOSKERNELS_SOURCE_SHA 6680347
KOKKOSKERNELS_TARGET_BRANCH develop
KOKKOSKERNELS_TARGET_REPO https://github.com/kokkos/kokkos-kernels
KOKKOSKERNELS_TARGET_SHA c3646ed
PR_LABELS bug;Cleanup
PULLREQUESTNUM 2008
TEST_REPO_ALIAS KOKKOSKERNELS

Build Information

Test Name: KokkosKernels_PullRequest_VEGA90A_Tpls_ROCM560

  • Build Num: 256
  • Status: PASSED

Jenkins Parameters

Parameter Name Value
KOKKOSKERNELS_SOURCE_REPO https://github.com/lucbv/kokkos-kernels
KOKKOSKERNELS_SOURCE_SHA 6680347
KOKKOSKERNELS_TARGET_BRANCH develop
KOKKOSKERNELS_TARGET_REPO https://github.com/kokkos/kokkos-kernels
KOKKOSKERNELS_TARGET_SHA c3646ed
PR_LABELS bug;Cleanup
PULLREQUESTNUM 2008
TEST_REPO_ALIAS KOKKOSKERNELS

@kokkos-devops-admin
Copy link

Status Flag 'Pre-Merge Inspection' - - This Pull Request Requires Inspection... The code must be inspected by a member of the Team before Testing/Merging
NO REVIEWS HAVE BEEN PERFORMED ON THIS PULL REQUEST!

@kokkos-devops-admin
Copy link

All Jobs Finished; status = PASSED, However Inspection must be performed before merge can occur...

2 similar comments
@kokkos-devops-admin
Copy link

All Jobs Finished; status = PASSED, However Inspection must be performed before merge can occur...

@kokkos-devops-admin
Copy link

All Jobs Finished; status = PASSED, However Inspection must be performed before merge can occur...

@kokkos-devops-admin
Copy link

Status Flag 'Pre-Merge Inspection' - SUCCESS: The last commit to this Pull Request has been INSPECTED AND APPROVED by [ brian-kelley ]!

@kokkos-devops-admin
Copy link

Status Flag 'Pull Request AutoTester' - Pull Request MUST BE MERGED MANUALLY BY Project Team - This Repo does not support Automerge

cwpearson
cwpearson approved these changes Oct 23, 2023
View reviewed changes
Copy link
Contributor

@cwpearson cwpearson left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Works for me, thanks!

@lucbv lucbv merged commit 4b5793b into kokkos:develop Oct 23, 2023
7 checks passed
@lucbv lucbv deleted the bspgemm_cusparse_hang branch October 23, 2023 19:49
@lucbv lucbv mentioned this pull request Oct 23, 2023
Closed
@ndellingwood
Copy link
Contributor

Candidate for cherry-pick to release-candidate-4.2.00?

@ndellingwood ndellingwood mentioned this pull request Oct 24, 2023
Closed
@cwpearson cwpearson mentioned this pull request Sep 9, 2024
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@cwpearson cwpearson cwpearson approved these changes

@brian-kelley brian-kelley brian-kelley approved these changes

@ndellingwood ndellingwood Awaiting requested review from ndellingwood

Assignees

@lucbv lucbv

Labels
bug Cleanup Code maintenance that isn't a bugfix or new feature
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
5 participants
@lucbv @kokkos-devops-admin @ndellingwood @brian-kelley @cwpearson