Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Warn and skip update-schema when deployed version > new version #3032

Closed
lmsurpre opened this issue Nov 22, 2021 · 7 comments
Closed

Warn and skip update-schema when deployed version > new version #3032

lmsurpre opened this issue Nov 22, 2021 · 7 comments
Assignees
Labels
enhancement New feature or request schema

Comments

@lmsurpre
Copy link
Member

lmsurpre commented Nov 22, 2021

Is your feature request related to a problem? Please describe.
In https://github.com/IBM/FHIR/pull/2880/files#diff-b4577e1635a5c980b9e41f30be0c6abcbf3fe5e6915b118fd319963bdac70a78R440 we added a check so that we skip --update-schema when the deployed version is already up-to-date.

However, this got implemented in SchemaVersionsManager as an == check: https://github.com/IBM/FHIR/pull/2880/files#diff-125e38fd75a71288d1716b5c3c920f85cdca913358ce3ec5b236973890ccddfcR101

This means that the updateSchema code will run if the currently-deployed schema version is greater than the new version being applied.
And because we always updated the stored procs (irregardless of version), this can have unintended consequences.

Describe the solution you'd like
If the currently-deployed schema version is greater than the one being "updated to", then log a warning and skip the work.
Most importantly, do NOT update (i.e. downgrade) the stored procs and do NOT update the whole-schema version.

Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.

Acceptance Criteria

  1. GIVEN [a precondition]
    AND [another precondition]
    WHEN [test step]
    AND [test step]
    THEN [verification step]
    AND [verification step]

Additional context
In one variation of this issue, you can end up with back-level stored proc definitions that don't update when you run --update-schema with a newer version of the tool.

  1. run --update-schema with version 4.10.0 of the schematool. this updates everything (including stored proc). the 4.10 application's healthcheck is happy because whole-schema version is 4.10
  2. run --update-schema with version 4.9.2 of the schematool (e.g. on accident). this does a bunch of no-op work AND updates the stored procs to the level they were at on 4.9.2 but whole-schema version is still 4.10
  3. healthcheck still passes on version 4.10 of the application, but the actual stored proc fails because the definition in the db is back-level
  4. running --update-schema with version 4.10.0 of the schematool does not fix it, because the whole-schema version says its at the latest version... its just that step 2 has accidentally downgraded the stored procs

If you run into this scenario, the fix is to execute the --update-proc (and not --update-schema) action of the 4.10 version of the tool.

@prb112 prb112 added enhancement New feature or request schema labels Nov 22, 2021
@d0roppe
Copy link
Collaborator

d0roppe commented Nov 22, 2021

With this issue we may also want to address the drop-schema (incorrect version used issue) that an error is issued if you try to drop a older schema with a newer tool, or vice versa. Maybe even tell them what version of the tool needs to be used.

@lmsurpre
Copy link
Member Author

A related scenario I thought of:

  1. User specifies a particular set of resourceTypes to deploy in the schema (this feature still isn't documented but its there... Specify list of include / exclude resources for the IBM FHIR Server schema #819)
  2. User wants to change the set of resoureTypes in the schema, but at the same schema version

Currently, there is no way to force it to do its thing when the schema version matches the expected schema version. For this use case, and maybe also for the case described in "Additional context" above, I think it would be nice to add a --force option which would bypass the version check and attempt to apply the schema update.

@punktilious punktilious self-assigned this Jan 14, 2022
@punktilious
Copy link
Collaborator

For the drop schema, I'm thinking we want to make it dictionary-driven so that it doesn't matter which version you have.

@punktilious
Copy link
Collaborator

The new --force option will only attempt to apply the schema update if the schema version actually matches the database version. This is important so that we don't inadvertently apply the wrong procedures. But this does allow us to bypass the version check when adding more resource types, as requested by @lmsurpre

punktilious added a commit that referenced this issue Jan 17, 2022
Signed-off-by: Robin Arnold <robin.arnold@ibm.com>
@d0roppe
Copy link
Collaborator

d0roppe commented Jan 17, 2022

Any issues with multi Tenant on different versions?

punktilious added a commit that referenced this issue Jan 17, 2022
Signed-off-by: Robin Arnold <robin.arnold@ibm.com>
punktilious added a commit that referenced this issue Jan 17, 2022
Signed-off-by: Robin Arnold <robin.arnold@ibm.com>
lmsurpre pushed a commit that referenced this issue Jan 19, 2022
…3200)

* issue #3032 only update when current schema is behind code version

Signed-off-by: Robin Arnold <robin.arnold@ibm.com>

* issue #3032 return error status when force arg given but does not apply

Signed-off-by: Robin Arnold <robin.arnold@ibm.com>

* issue #3032 added force option to menu help

Signed-off-by: Robin Arnold <robin.arnold@ibm.com>
@lmsurpre
Copy link
Member Author

to QA:

  1. create a schema at level V26 (e.g. via R4B)
  2. run the main version to "update" the schem to V25...it shouldn't try to update since the db version is past the latest code version

@d0roppe
Copy link
Collaborator

d0roppe commented Feb 21, 2022

the update-schema does not do any updates when the schema version > attempted update schema version. Closing issue.

> java -jar ./fhir-persistence-schema-*-SNAPSHOT-cli.jar --prop-file local.properties --schema-name fhirdata --update-schema --db-type postgresql
2022-02-21 10:33:08.323 00000001    INFO .common.JdbcConnectionProvider Opening connection to database: jdbc:postgresql://localhost:7000/fhir
2022-02-21 10:33:08.586 00000001    INFO base.utils.schema.LeaseManager Requesting update lease for schema 'FHIRDATA' [attempt 1]
2022-02-21 10:33:08.611 00000001    INFO   com.ibm.fhir.schema.app.Main Schema is up-to-date; skipping update for: 'FHIRDATA'
2022-02-21 10:33:08.611 00000001    INFO base.utils.schema.LeaseManager Canceling lease for schema 'FHIRDATA'
2022-02-21 10:33:08.612 00000022    INFO base.utils.schema.LeaseManager Lease maintenance loop terminated
2022-02-21 10:33:08.612 00000001    INFO base.utils.schema.LeaseManager Lease maintenance thread terminated
2022-02-21 10:33:08.617 00000001    INFO base.utils.schema.LeaseManager Lease canceled for schema 'FHIRDATA'
2022-02-21 10:33:08.620 00000001    INFO base.utils.schema.LeaseManager Requesting update lease for schema 'FHIR_OAUTH' [attempt 1]
2022-02-21 10:33:08.639 00000001    INFO   com.ibm.fhir.schema.app.Main Schema is current; skipping update for: 'FHIR_OAUTH'
2022-02-21 10:33:08.640 00000001    INFO base.utils.schema.LeaseManager Canceling lease for schema 'FHIR_OAUTH'
2022-02-21 10:33:08.640 00000023    INFO base.utils.schema.LeaseManager Lease maintenance loop terminated
2022-02-21 10:33:08.641 00000001    INFO base.utils.schema.LeaseManager Lease maintenance thread terminated
2022-02-21 10:33:08.647 00000001    INFO base.utils.schema.LeaseManager Lease canceled for schema 'FHIR_OAUTH'
2022-02-21 10:33:08.651 00000001    INFO base.utils.schema.LeaseManager Requesting update lease for schema 'FHIR_JBATCH' [attempt 1]
2022-02-21 10:33:08.671 00000001    INFO   com.ibm.fhir.schema.app.Main Schema is current; skipping update for: 'FHIR_JBATCH'
2022-02-21 10:33:08.671 00000001    INFO base.utils.schema.LeaseManager Canceling lease for schema 'FHIR_JBATCH'
2022-02-21 10:33:08.672 00000024    INFO base.utils.schema.LeaseManager Lease maintenance loop terminated
2022-02-21 10:33:08.672 00000001    INFO base.utils.schema.LeaseManager Lease maintenance thread terminated
2022-02-21 10:33:08.676 00000001    INFO base.utils.schema.LeaseManager Lease canceled for schema 'FHIR_JBATCH'
2022-02-21 10:33:08.681 00000001    INFO   com.ibm.fhir.schema.app.Main Processing took:   0.385 s
2022-02-21 10:33:08.681 00000001    INFO   com.ibm.fhir.schema.app.Main SCHEMA CHANGE: OK

@d0roppe d0roppe closed this as completed Feb 21, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request schema
Projects
None yet
Development

No branches or pull requests

4 participants