Skip to content

Commit

Permalink
stabilize unreliable tests for smoother CI operation.
Browse files Browse the repository at this point in the history
wrap platform/core.windows, favorites and settings into retry script.

mitigates the following exception which causes the tests to be very
unreliable in CI:
java.lang.AssertionError: Should have found a nonproblematic provider
of needs org.openide.modules.jre.JavaFX among [...]

Separate reliable from unreliable PHP tests and wrap unreliable tests
in retry script.
  • Loading branch information
mbien committed Jul 6, 2023
1 parent 5f9b451 commit 3a1176e
Show file tree
Hide file tree
Showing 2 changed files with 16 additions and 6 deletions.
12 changes: 8 additions & 4 deletions .github/workflows/main.yml
Original file line number Diff line number Diff line change
Expand Up @@ -934,7 +934,7 @@ jobs:
run: ant $OPTS -Dvanilla.javac.exists=true -f platform/core.ui test

- name: platform/core.windows
run: ant $OPTS -Dvanilla.javac.exists=true -f platform/core.windows test
run: .github/retry.sh ant $OPTS -Dvanilla.javac.exists=true -f platform/core.windows test

- name: platform/editor.mimelookup
run: ant $OPTS -Dvanilla.javac.exists=true -f platform/editor.mimelookup test
Expand All @@ -943,7 +943,7 @@ jobs:
run: ant $OPTS -Dvanilla.javac.exists=true -f platform/editor.mimelookup.impl test

- name: platform/favorites
run: ant $OPTS -Dvanilla.javac.exists=true -f platform/favorites test
run: .github/retry.sh ant $OPTS -Dvanilla.javac.exists=true -f platform/favorites test

- name: platform/javahelp
run: ant $OPTS -Dvanilla.javac.exists=true -f platform/javahelp test-unit
Expand Down Expand Up @@ -1143,7 +1143,7 @@ jobs:
run: ant $OPTS -Dvanilla.javac.exists=true -f platform/sendopts test

- name: platform/settings
run: ant $OPTS -Dvanilla.javac.exists=true -f platform/settings test
run: .github/retry.sh ant $OPTS -Dvanilla.javac.exists=true -f platform/settings test

- name: platform/spi.actions
run: ant $OPTS -Dvanilla.javac.exists=true -f platform/spi.actions test
Expand Down Expand Up @@ -2234,7 +2234,11 @@ jobs:
# longest step (~40min)
- name: php.editor
if: env.test_php == 'true' && success()
run: ant $OPTS -f php/php.editor test
run: ant $OPTS -Dtest.config=stable -f php/php.editor test

- name: php.editor (unreliable tests)
if: env.test_php == 'true' && success()
run: .github/retry.sh ant $OPTS -Dtest.config=unreliable -f php/php.editor test

- name: php.latte
run: ant $OPTS -f php/php.latte test
Expand Down
10 changes: 8 additions & 2 deletions php/php.editor/nbproject/project.properties
Original file line number Diff line number Diff line change
Expand Up @@ -30,9 +30,15 @@ test.config.uicommit.includes=\
# org/netbeans/test/php/Commit.class,\
# org/netbeans/test/php/project/test*.class

test.config.stable.includes=\
org/netbeans/test/php/*/test*.class
# known to fail sporadically, should be tested separately in CI, ideally with retry script until stabilized
test.config.unreliable.includes=\
org/netbeans/modules/php/editor/verification/IntroduceSuggestionTest.class,\
org/netbeans/modules/php/editor/csl/GotoDeclarationPHP81Test.class,\
org/netbeans/modules/php/editor/csl/FoldingTest.class
test.config.unreliable.excludes=

test.config.stable.includes=**/*Test.class
test.config.stable.excludes=${test.config.unreliable.includes}

test.config.normal.includes=\
org/netbeans/test/php/Commit.class,\
Expand Down

0 comments on commit 3a1176e

Please sign in to comment.