Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

End-to-end automation test for reindexing #1751

Closed
lmsurpre opened this issue Nov 30, 2020 · 4 comments
Closed

End-to-end automation test for reindexing #1751

lmsurpre opened this issue Nov 30, 2020 · 4 comments
Assignees
Labels
automation automation P2 Priority 2 - Should Have persistence testing

Comments

@lmsurpre
Copy link
Member

lmsurpre commented Nov 30, 2020

We released 4.5.0 without catching a reindex bug that prevented the correct migration of data: #1742

To prevent that from happening again, we should include a test in our CI pipeline which does the following:

  1. writes a resource / some set of resources (maybe the Basic ones used in our persistence search tests?)
  2. changes the configured set of search parameters (maybe add the corresponding extension-search-parameters for the search tests?)
  3. invoke reindex (as needed until everything is refreshed)
  4. performs a search / set of searches to verify the data is returned as expected

For even greater assurance, we could extend the existing db migration test to do the following:

  1. deploy an older version of the schema
  2. write a resource / some set of resources (maybe the Basic ones used in our persistence search tests?)
  3. apply the schema migration
  4. (optional) adjust the search parameters slightly
  5. invoke reindex
  6. perform a search to verify the expected resources are still returned

UPDATE: we probably don't need to tie the re-index test to the migration test. Having them as two separate tests should be sufficient.

@lmsurpre lmsurpre changed the title Extend migration test to include data migration and reindexing End-to-end automation test for reindexing Feb 8, 2021
@lmsurpre lmsurpre added the automation automation label Feb 8, 2021
@prb112 prb112 added the testing label Feb 8, 2021
@lmsurpre
Copy link
Member Author

Does https://github.com/IBM/FHIR/pull/2107/files sufficiently cover this one?

@prb112
Copy link
Contributor

prb112 commented Mar 25, 2021

Does https://github.com/IBM/FHIR/pull/2107/files sufficiently cover this one?

Currently, it's just a sniff test. Does it work with one resource.
If you are OK, we can rock with one sniff test (across derby, postgres, db2) or we can add a small loop of say one hundred.

Also, to be a true test, we would create a set of search parameter, search find the resource by search parameters related to the types, and then change the search parameters (not type) to a new FHIR Path, and show they are not retrievable by the search parameter.

@lmsurpre
Copy link
Member Author

OK, thanks Paul. Lets leave this one open then for the testing improvements you mention.

@lmsurpre lmsurpre added the P2 Priority 2 - Should Have label May 24, 2021
@lmsurpre lmsurpre added this to the Sprint 2021-08 milestone Jun 1, 2021
prb112 added a commit that referenced this issue Jun 8, 2021
Signed-off-by: Paul Bastide <pbastide@us.ibm.com>
lmsurpre added a commit that referenced this issue Jun 14, 2021
* End-to-end automation test for reindexing #1751

Signed-off-by: Paul Bastide <pbastide@us.ibm.com>

* Update reindex operation tests

Signed-off-by: Paul Bastide <pbastide@us.ibm.com>

* Refine the Scripts

Signed-off-by: Paul Bastide <pbastide@us.ibm.com>

* Update CI

Signed-off-by: Paul Bastide <pbastide@us.ibm.com>

* Implement Reindex IT Matrix for Db2

Signed-off-by: Paul Bastide <pbastide@us.ibm.com>

* Add Derby and Postgres to the Matrix

Signed-off-by: Paul Bastide <pbastide@us.ibm.com>

* Update json

Signed-off-by: Paul Bastide <pbastide@us.ibm.com>

* Update pre req

Signed-off-by: Paul Bastide <pbastide@us.ibm.com>

* Update hostname

Signed-off-by: Paul Bastide <pbastide@us.ibm.com>

* Update the Derby Build

Signed-off-by: Paul Bastide <pbastide@us.ibm.com>

* Update for docker-compose vs docker compose

Signed-off-by: Paul Bastide <pbastide@us.ibm.com>

* Update to use the right docker

Signed-off-by: Paul Bastide <pbastide@us.ibm.com>

* Update Reindex Operation

Signed-off-by: Paul Bastide <pbastide@us.ibm.com>

* Update

Signed-off-by: Paul Bastide <pbastide@us.ibm.com>

* Update

Signed-off-by: Paul Bastide <pbastide@us.ibm.com>

* Confirming Test Fails

Signed-off-by: Paul Bastide <pbastide@us.ibm.com>

* Confirming Test Fails

Signed-off-by: Paul Bastide <pbastide@us.ibm.com>

* Update

Signed-off-by: Paul Bastide <pbastide@us.ibm.com>

* Update Reindex Operation

Signed-off-by: Paul Bastide <pbastide@us.ibm.com>

* Update

Signed-off-by: Paul Bastide <pbastide@us.ibm.com>

* Disables Nats in the pre-integration-test and hardens gather-logs.sh

Signed-off-by: Paul Bastide <pbastide@us.ibm.com>

* Update Docker Compose for Db2 to add Grant-To

Signed-off-by: Paul Bastide <pbastide@us.ibm.com>

* Update for db2 tenant-key

Signed-off-by: Paul Bastide <pbastide@us.ibm.com>

* Update the Build to Support a dynamic tenant-key

Signed-off-by: Paul Bastide <pbastide@us.ibm.com>

* Reindex Does not Document the ResourceLogicalId behavior in the Search Guide #2494

Signed-off-by: Paul Bastide <pbastide@us.ibm.com>

* Reindex Returns 500 when the Client sends a bad timestamp #2493

Signed-off-by: Paul Bastide <pbastide@us.ibm.com>

* Update integration-tests so that I can log the errors

Signed-off-by: Paul Bastide <pbastide@us.ibm.com>

* Update Reindex Operation

Signed-off-by: Paul Bastide <pbastide@us.ibm.com>

* Build the Right Image

Signed-off-by: Paul Bastide <pbastide@us.ibm.com>

* Update Phase1

Signed-off-by: Paul Bastide <pbastide@us.ibm.com>

* Update for proper bundle

Signed-off-by: Paul Bastide <pbastide@us.ibm.com>

* Update Reindex

Signed-off-by: Paul Bastide <pbastide@us.ibm.com>

* Add conditional restart to db2

Signed-off-by: Paul Bastide <pbastide@us.ibm.com>

* Update to use a startup script

Signed-off-by: Paul Bastide <pbastide@us.ibm.com>

* Various Updates

- Update BasicDate.json to support code vs valueString
- Improve Test Coverage for ReindexOperation

Signed-off-by: Paul Bastide <pbastide@us.ibm.com>

* Disable Default Tests

Signed-off-by: Paul Bastide <pbastide@us.ibm.com>

* Update operation/fhir-operation-reindex/src/main/java/com/ibm/fhir/operation/reindex/ReindexOperation.java

Signed-off-by: Paul Bastide <pbastide@us.ibm.com>

Co-authored-by: Lee Surprenant <lmsurpre@us.ibm.com>

* Update per Code Review

Signed-off-by: Paul Bastide <pbastide@us.ibm.com>

* Adds support for Type,Resource and update System-Type-Instance handling
in FHIROperationRegistry

Signed-off-by: Paul Bastide <pbastide@us.ibm.com>

* Update Docs

Signed-off-by: Paul Bastide <pbastide@us.ibm.com>

* Update reindex workflow and phase2

Signed-off-by: Paul Bastide <pbastide@us.ibm.com>

* Update per Code Review

Signed-off-by: Paul Bastide <pbastide@us.ibm.com>

Co-authored-by: Lee Surprenant <lmsurpre@us.ibm.com>
@d0roppe
Copy link
Collaborator

d0roppe commented Jul 1, 2021

Tests are part of the integration tests that run as part of the build now. closing issue

@d0roppe d0roppe closed this as completed Jul 1, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
automation automation P2 Priority 2 - Should Have persistence testing
Projects
None yet
Development

No branches or pull requests

3 participants