Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

use run_shell_cmd in custom easyblock for ABAQUS #3414

Merged
merged 8 commits into from
Aug 29, 2024
Merged

use run_shell_cmd in custom easyblock for ABAQUS #3414

merged 8 commits into from
Aug 29, 2024

Conversation

boegel
Copy link
Member

@boegel boegel commented Aug 16, 2024

No description provided.

@boegel boegel added change EasyBuild-5.0 EasyBuild 5.0 labels Aug 16, 2024
@boegel boegel added this to the 5.0 milestone Aug 16, 2024
@boegel boegel marked this pull request as draft August 17, 2024 08:10
@boegel
use \s rather than space in [ ] in part of question pattern used in A…
b7f6ca2 
…BAQUS easyblock, since spaces get replaced with [\s\n]+ by run_shell_cmd
@boegel
Copy link
Member Author

boegel commented Aug 20, 2024

Hitting an interesting problem when testing with ABAQUS-2022-hotfix-2214.eb.

At some point run_shell_cmd checks for question pattern matches in following end of stdout:

== 2024-08-20 21:10:00,492 run.py:270 DEBUG Found match for question pattern '\s*(?P<nr>[-0-9]+)[\s\n]+\[\*\][\s\n]+Isight[\s\w]*\n((?!Enter[\s\n]+selection[\s\n]+\(default:[\s\n]+Next\):\s*)(?!___)[\S[\s\n]+]*\n)*Enter[\s\n]+selection[\s\n]+\(default:[\s\n]+Next\):\s*$[\s\n]*$' at end of stdout: lished Products CAA API
  7 [*] Isight

Enter selection (default: Next):
 -1 Select All
  0 Select None
  1 [ ] NETVIBES Exalead CloudView
  2 [ ] EXALEAD Search for Documentation
  3 [ ] Extended Product Documentation
  4 [ ] FLEXnet License Server
  5 [*] SIMULIA Established Products
  6 [*] SIMULIA Established Products CAA API
  7 [ ] Isight

Enter selection (default: Next):
 -1 Select All
  0 Select None
  1 [ ] NETVIBES Exalead CloudView
  2 [ ] EXALEAD Search for Documentation
  3 [ ] Extended Product Documentation
  4 [ ] FLEXnet License Server
  5 [*] SIMULIA Established Products
  6 [*] SIMULIA Established Products CAA API
  7 [*] Isight

Enter selection (default: Next):
 -1 Select All
  0 Select None
  1 [ ] NETVIBES Exalead CloudView
  2 [ ] EXALEAD Search for Documentation
  3 [ ] Extended Product Documentation
  4 [ ] FLEXnet License Server
  5 [*] SIMULIA Established Products
  6 [*] SIMULIA Established Products CAA API
  7 [ ] Isight

Enter selection (default: Next):

The problem here is that "\[\*\] Isight" matches, even though it's not at the end of the output.
It should be matched once, to disable the installation of Isight.
But now we're going into an infinite enable/disable loop, because run_shell_cmd considers the full output, not just the last part of the output...

This probably indicates that another bug fix is required after easybuilders/easybuild-framework#4617, to only consider the new part of the output after a question has been answered?

@branfosj branfosj mentioned this pull request Aug 21, 2024
Closed
@boegel
fix issues with regex patterns for questions raised in interactive in…
6472d46 
…stallation command of ABAQUS, by using non-greedy matching (*?) in combination with negative lookahead (?!)
@boegel boegel marked this pull request as ready for review August 28, 2024 20:14
@boegel
Copy link
Member Author

boegel commented Aug 28, 2024 •
edited
Loading

Problem with the tests has nothing to do with the changes to the ABAQUS easyblock, it's fallout from a change in framework that broke the easyblocks test suite (I'll get that fixed in a separate PR):

edit: fixed by:

@boegel boegel force-pushed the abaqus_run_shell_cmd branch from d0e941a to 58599f7 Compare August 28, 2024 20:20
@boegel
Copy link
Member Author

boegel commented Aug 28, 2024

Test report by @boegel

Overview of tested easyconfigs (in order)

  • SUCCESS ABAQUS-2022-hotfix-2214.eb

Build succeeded for 1 out of 1 (1 easyconfigs in total)
node3515.doduo.os - Linux RHEL 8.8, x86_64, AMD EPYC 7552 48-Core Processor, Python 3.6.8
See https://gist.github.com/boegel/05fca2a4aafc3660f0ad6dd2878a33bc for a full test report.

@boegel boegel mentioned this pull request Aug 28, 2024
Merged
@Micket
Copy link
Contributor

Micket commented Aug 28, 2024

Test report by @Micket

Overview of tested easyconfigs (in order)

Build succeeded for 0 out of 1 (1 easyconfigs in total)
alvis-skylake-build - Linux Rocky Linux 8.9, x86_64, Intel Xeon Processor (Skylake, IBRS, no TSX), Python 3.6.8
See https://gist.github.com/Micket/f15b99bfca3191a72ff3bdf981ea8e43 for a full test report.

@Micket
Copy link
Contributor

Micket commented Aug 28, 2024

Test report by @Micket

Overview of tested easyconfigs (in order)

  • SUCCESS ABAQUS-2022-hotfix-2223.eb

Build succeeded for 1 out of 1 (1 easyconfigs in total)
alvis-skylake-build - Linux Rocky Linux 8.9, x86_64, Intel Xeon Processor (Skylake, IBRS, no TSX), Python 3.6.8
See https://gist.github.com/Micket/cfd93697b9f9af61cecf227c8a8297be for a full test report.

@branfosj
Copy link
Member

Test report by @branfosj

Overview of tested easyconfigs (in order)

  • SUCCESS ABAQUS-2021.eb

Build succeeded for 1 out of 1 (1 easyconfigs in total)
bear-pg0105u03a - Linux RHEL 8.6, x86_64, Intel(R) Xeon(R) Platinum 8360Y CPU @ 2.40GHz (icelake), Python 3.6.8
See https://gist.github.com/branfosj/86fac84a70f2d1e356b660e403e9e471 for a full test report.

@branfosj
Copy link
Member

Test report by @branfosj

Overview of tested easyconfigs (in order)

  • SUCCESS ABAQUS-2024-hotfix-2405.eb

Build succeeded for 1 out of 1 (1 easyconfigs in total)
bear-pg0105u03b - Linux RHEL 8.6, x86_64, Intel(R) Xeon(R) Platinum 8360Y CPU @ 2.40GHz (icelake), Python 3.6.8
See https://gist.github.com/branfosj/61d9dbdfbb3d4090f27573ee2cf4b226 for a full test report.

@Micket
Copy link
Contributor

Micket commented Aug 29, 2024

Test report by @Micket

Overview of tested easyconfigs (in order)

Build succeeded for 0 out of 1 (1 easyconfigs in total)
alvis-skylake-build - Linux Rocky Linux 8.9, x86_64, Intel Xeon Processor (Skylake, IBRS, no TSX), Python 3.6.8
See https://gist.github.com/Micket/c8ac16672d415e8d38dc0219e1503d03 for a full test report.

@branfosj
Copy link
Member

Test report by @branfosj

Overview of tested easyconfigs (in order)

  • SUCCESS ABAQUS-2023-hotfix-2314.eb

Build succeeded for 1 out of 1 (1 easyconfigs in total)
bear-pg0105u03a - Linux RHEL 8.6, x86_64, Intel(R) Xeon(R) Platinum 8360Y CPU @ 2.40GHz (icelake), Python 3.6.8
See https://gist.github.com/branfosj/98d04f891fa477887b8cf7c910f7b5c3 for a full test report.

@branfosj
Copy link
Member

qa_timeout=100 is the default value. We should increase any that need increasing and remove the others.

Micket
Micket approved these changes Aug 29, 2024
View reviewed changes
Copy link
Contributor

@Micket Micket left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM (I'm fine with 1000 everywhere. Probably overkill for many of these, but who knows, still isn't to terrible to wait out 1000 if there truly is some situation where it is broken)

@boegel
Copy link
Member Author

boegel commented Aug 29, 2024

Test report by @boegel

Overview of tested easyconfigs (in order)

  • SUCCESS ABAQUS-2022-hotfix-2214.eb

Build succeeded for 1 out of 1 (1 easyconfigs in total)
node3515.doduo.os - Linux RHEL 8.8, x86_64, AMD EPYC 7552 48-Core Processor, Python 3.6.8
See https://gist.github.com/boegel/3d7362b49834df6f2a15c0cddbd27bb3 for a full test report.

@branfosj
Copy link
Member

Test report by @branfosj

Overview of tested easyconfigs (in order)

  • SUCCESS ABAQUS-2021.eb

Build succeeded for 1 out of 1 (1 easyconfigs in total)
bear-pg0105u03a - Linux RHEL 8.6, x86_64, Intel(R) Xeon(R) Platinum 8360Y CPU @ 2.40GHz (icelake), Python 3.6.8
See https://gist.github.com/branfosj/fda52432711b771d7164d3a2b418cb66 for a full test report.

@branfosj
Copy link
Member

Going in, thanks @boegel!

@branfosj branfosj merged commit 068d8ea into easybuilders:5.0.x Aug 29, 2024
19 checks passed
@branfosj
Copy link
Member

Test report by @branfosj

Overview of tested easyconfigs (in order)

  • SUCCESS ABAQUS-2023-hotfix-2314.eb

Build succeeded for 1 out of 1 (1 easyconfigs in total)
bear-pg0105u03a - Linux RHEL 8.6, x86_64, Intel(R) Xeon(R) Platinum 8360Y CPU @ 2.40GHz (icelake), Python 3.6.8
See https://gist.github.com/branfosj/9eb9b2ac406667c4d67fc60126b140e5 for a full test report.

@branfosj
Copy link
Member

Test report by @branfosj

Overview of tested easyconfigs (in order)

  • SUCCESS ABAQUS-2024-hotfix-2405.eb

Build succeeded for 1 out of 1 (1 easyconfigs in total)
bear-pg0105u03a - Linux RHEL 8.6, x86_64, Intel(R) Xeon(R) Platinum 8360Y CPU @ 2.40GHz (icelake), Python 3.6.8
See https://gist.github.com/branfosj/deda4dc2ff7a74383102bc64a264b00c for a full test report.

@boegel boegel deleted the abaqus_run_shell_cmd branch August 29, 2024 15:47
@boegel boegel mentioned this pull request Dec 18, 2024
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@branfosj branfosj branfosj left review comments

@Micket Micket Micket approved these changes

Assignees
No one assigned
Labels
change EasyBuild-5.0 EasyBuild 5.0
Projects
None yet
Milestone
5.0
Development

Successfully merging this pull request may close these issues.
3 participants
@boegel @Micket @branfosj