Skip to content
This repository has been archived by the owner on Jan 18, 2021. It is now read-only.

[Tests-Only] Implement separate expected-failures files for OC and EOS storage #444

Merged
merged 1 commit into from
Aug 18, 2020
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .drone.star
Original file line number Diff line number Diff line change
Expand Up @@ -122,7 +122,7 @@ def coreApiTests(ctx, coreBranch = 'master', coreCommit = '', part_number = 1, n
'BEHAT_FILTER_TAGS': '~@notToImplementOnOCIS&&~@toImplementOnOCIS&&~@preview-extension-required',
'DIVIDE_INTO_NUM_PARTS': number_of_parts,
'RUN_PART': part_number,
'EXPECTED_FAILURES_FILE': '/drone/src/tests/acceptance/expected-failures.txt'
'EXPECTED_FAILURES_FILE': '/drone/src/tests/acceptance/expected-failures-on-OC-storage.txt'
},
'commands': [
'cd /srv/app/testrunner',
Expand Down
5 changes: 3 additions & 2 deletions docs/testing.md
Original file line number Diff line number Diff line change
Expand Up @@ -97,8 +97,9 @@ To run a single test add `BEHAT_FEATURE=<feature file>` and specify the path to
### use existing tests for BDD

As a lot of scenarios are written for oC10, we can use those tests for Behaviour driven development in ocis.
Every scenario that does not work in OCIS, is listed in `tests/acceptance/expected-failures.txt` with a link to the related issue.
Those scenarios are run in the ordinary acceptance test pipeline in CI. The sccenarios that fail are checked against the
Every scenario that does not work in OCIS with OC storage, is listed in `tests/acceptance/expected-failures-on-OC-storage.txt` with a link to the related issue.
Similarly, scenarios that do not work in OCIS with EOS storage are listed in `tests/acceptance/expected-failures-on-EOS-storage.txt`.
Scenarios from the oC10 API acceptance tests are run in the ordinary acceptance test pipeline in CI. The scenarios that fail are checked against the
expected failures. If there are any differences then the CI pipeline fails.

Additionally, some issues have scenarios that demonstrate the current buggy behaviour in ocis(reva).
Expand Down
Original file line number Diff line number Diff line change
@@ -1,4 +1,7 @@
# this file contains the scenarios from ownCloud10 core API tests that are currently expected to fail
# when run with EOS storage
#
# ToDo: enable testing with EOS in CI and adjust this list to match what actually fails
#
# test scenarios that fail with OC storage (that were tagged skipOnOcis-OC-Storage in core)
#
Expand Down
Loading