Skip to content
This repository has been archived by the owner on Jan 18, 2021. It is now read-only.

Commit

Permalink
Implement separate expected-failures files for OC and EOS storage
Browse files Browse the repository at this point in the history
  • Loading branch information
phil-davis committed Aug 18, 2020
1 parent 55fccdb commit 719c96c
Show file tree
Hide file tree
Showing 4 changed files with 923 additions and 3 deletions.
2 changes: 1 addition & 1 deletion .drone.star
Original file line number Diff line number Diff line change
Expand Up @@ -122,7 +122,7 @@ def coreApiTests(ctx, coreBranch = 'master', coreCommit = '', part_number = 1, n
'BEHAT_FILTER_TAGS': '~@notToImplementOnOCIS&&~@toImplementOnOCIS&&~@preview-extension-required',
'DIVIDE_INTO_NUM_PARTS': number_of_parts,
'RUN_PART': part_number,
'EXPECTED_FAILURES_FILE': '/drone/src/tests/acceptance/expected-failures.txt'
'EXPECTED_FAILURES_FILE': '/drone/src/tests/acceptance/expected-failures-on-OC-storage.txt'
},
'commands': [
'cd /srv/app/testrunner',
Expand Down
5 changes: 3 additions & 2 deletions docs/testing.md
Original file line number Diff line number Diff line change
Expand Up @@ -97,8 +97,9 @@ To run a single test add `BEHAT_FEATURE=<feature file>` and specify the path to
### use existing tests for BDD

As a lot of scenarios are written for oC10, we can use those tests for Behaviour driven development in ocis.
Every scenario that does not work in OCIS, is listed in `tests/acceptance/expected-failures.txt` with a link to the related issue.
Those scenarios are run in the ordinary acceptance test pipeline in CI. The sccenarios that fail are checked against the
Every scenario that does not work in OCIS with OC storage, is listed in `tests/acceptance/expected-failures-on-OC-storage.txt` with a link to the related issue.
Similarly, scenarios that do not work in OCIS with EOS storage are listed in `tests/acceptance/expected-failures-on-EOS-storage.txt`.
Scenarios from the oC10 API acceptance tests are run in the ordinary acceptance test pipeline in CI. The scenarios that fail are checked against the
expected failures. If there are any differences then the CI pipeline fails.

Additionally, some issues have scenarios that demonstrate the current buggy behaviour in ocis(reva).
Expand Down
Original file line number Diff line number Diff line change
@@ -1,4 +1,7 @@
# this file contains the scenarios from ownCloud10 core API tests that are currently expected to fail
# when run with EOS storage
#
# ToDo: enable testing with EOS in CI and adjust this list to match what actually fails
#
# test scenarios that fail with OC storage (that were tagged skipOnOcis-OC-Storage in core)
#
Expand Down
Loading

0 comments on commit 719c96c

Please sign in to comment.