Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Citus distribution support and async/remote indexing #3683

Merged
merged 45 commits into from
Jun 21, 2022
Merged

Citus distribution support and async/remote indexing #3683

merged 45 commits into from
Jun 21, 2022

Conversation

punktilious
Copy link
Collaborator

No description provided.

punktilious added 30 commits March 7, 2022 11:29
@punktilious
issue #3437 experimental support for Citus database
c4a11be 
Signed-off-by: Robin Arnold <robin.arnold@ibm.com>
@punktilious
issue #3437 schema build changes to support Citus distribution
7244dce 
Signed-off-by: Robin Arnold <robin.arnold@ibm.com>
@punktilious
issue #3437 citus requires some custom ingestion logic
a7f2379 
Signed-off-by: Robin Arnold <robin.arnold@ibm.com>
@punktilious
issue #3437 distribute add_any_resource function in Citus
4398119 
Signed-off-by: Robin Arnold <robin.arnold@ibm.com>
@punktilious
issue #3437 merge latest main
d0eaa36 
Signed-off-by: Robin Arnold <robin.arnold@ibm.com>
@punktilious
Merge remote-tracking branch 'origin/HEAD' into robin-citus-1
8a237ad
@punktilious
Merge remote-tracking branch 'origin/HEAD' into robin-citus-1
10df2e8
@punktilious
issue #3437 distributed schema using shard_key
68819ab 
Signed-off-by: Robin Arnold <robin.arnold@ibm.com>
@punktilious
issue #3437 remote index kafka consumer
0752bbb 
Signed-off-by: Robin Arnold <robin.arnold@ibm.com>
@punktilious
issue #3437 remote index support for profile, tags and security
6343ab9 
Signed-off-by: Robin Arnold <robin.arnold@ibm.com>
@punktilious
issue #3437 remote index wait until server transaction commits
a8acdb8 
Signed-off-by: Robin Arnold <robin.arnold@ibm.com>
@punktilious
issue #3437 add logical_resource_ident and ref param support
7125f67 
Signed-off-by: Robin Arnold <robin.arnold@ibm.com>
@punktilious
issue #3437 fix reference unit tests for new ref_values table
f705e5a 
Signed-off-by: Robin Arnold <robin.arnold@ibm.com>
@punktilious
issue #3437 use logical_resource_ident for absolute references
0c1702b 
Signed-off-by: Robin Arnold <robin.arnold@ibm.com>
@punktilious
issue #3437 compilation issues from shard update
8c5cc75 
Signed-off-by: Robin Arnold <robin.arnold@ibm.com>
@punktilious
issue #3437 fix remote index processing for new logical_resource_iden…
ca0bfa0 
…t recs

Signed-off-by: Robin Arnold <robin.arnold@ibm.com>
@punktilious
issue #3437 tidied javadoc
246897f 
Signed-off-by: Robin Arnold <robin.arnold@ibm.com>
@punktilious
issue #3437 fixed issues using Citus
24bb883 
Signed-off-by: Robin Arnold <robin.arnold@ibm.com>
@punktilious
issue #3437 use identity cache for more ids
3e6699d 
Signed-off-by: Robin Arnold <robin.arnold@ibm.com>
@punktilious
issue #3437 merge with main for R4B
19daa4c 
Signed-off-by: Robin Arnold <robin.arnold@ibm.com>
@punktilious
issue #3437 do not apply permissions to tables not created
fcf2c1e 
Signed-off-by: Robin Arnold <robin.arnold@ibm.com>
@punktilious
issue #3437 initial derby unit test for fhir-remote-index
f2687cc 
Signed-off-by: Robin Arnold <robin.arnold@ibm.com>
@punktilious
issue #3437 more unit test coverage for fhir-remote-index
f44b4cb 
Signed-off-by: Robin Arnold <robin.arnold@ibm.com>
@punktilious
issue #3437 delete parameters from new xx_ref_values
4de35a5 
Signed-off-by: Robin Arnold <robin.arnold@ibm.com>
@punktilious
issue #3437 allow reindex to be forced after database migration
40e2454 
Signed-off-by: Robin Arnold <robin.arnold@ibm.com>
@punktilious
issue #3437 use logical_resource_ident for read and vread with citus
c3ab574 
Signed-off-by: Robin Arnold <robin.arnold@ibm.com>
@punktilious
issue #3437 fixed postgres and db2 add_any_resource to use lr ident
fe03611 
Signed-off-by: Robin Arnold <robin.arnold@ibm.com>
@punktilious
issue #3437 db-type arg now required when running schema tool
1644408 
Signed-off-by: Robin Arnold <robin.arnold@ibm.com>
@punktilious
issue #3437 fixed schema migration for Db2
7ca7633 
Signed-off-by: Robin Arnold <robin.arnold@ibm.com>
@punktilious
issue #3437 fixed missing grant for Db2
89e14ad 
Signed-off-by: Robin Arnold <robin.arnold@ibm.com>
@lmsurpre lmsurpre added the new-module Flag for identifying PRs that introduce a new module (and therefor require manual work for release) label Jun 13, 2022
@lmsurpre
Copy link
Member

note to self: we need to add the fhir-remote-index module to the list of cli tools. possibly it should get its on linux container image too.

@punktilious
issue #3697 documentation for fhir-remote-index
9d96843 
Signed-off-by: Robin Arnold <robin.arnold@ibm.com>
@punktilious
issue #3698 document schema changes incl citus support
451e804 
Signed-off-by: Robin Arnold <robin.arnold@ibm.com>
@punktilious
issue #3437 addressed review comments. Support whole system token val…
7a9221a 
…ues. Remove ref_version_id

Signed-off-by: Robin Arnold <robin.arnold@ibm.com>
@punktilious
issue #3437 fixing drop column for ref_version_id
dec8d3a 
Signed-off-by: Robin Arnold <robin.arnold@ibm.com>
@punktilious
issue #3437 tab to space and fixed comments
dd08e87 
Signed-off-by: Robin Arnold <robin.arnold@ibm.com>
@punktilious
issue #3437 fixed fhir-bucket and corrected DropView comment
f37233b 
Signed-off-by: Robin Arnold <robin.arnold@ibm.com>
@punktilious
issue #3437 corrected DropView comment
5f7e25d 
Signed-off-by: Robin Arnold <robin.arnold@ibm.com>
@punktilious
issue #3437 javadoc review comments
150e3c3 
Signed-off-by: Robin Arnold <robin.arnold@ibm.com>
punktilious and others added 4 commits June 17, 2022 10:28
@punktilious
issue #3437 skip parameter storage for remote index with derby and db2
5aa9325 
Signed-off-by: Robin Arnold <robin.arnold@ibm.com>
@punktilious
issue #3437 add instanceIdentifier to remote index messages and other…
c446a15 
… review fixes

Signed-off-by: Robin Arnold <robin.arnold@ibm.com>
@punktilious
issue #3713 use distributed add_any_resource function for citus (#3718)
583903a 
* issue #3713 use distributed add_any_resource function for citus

Signed-off-by: Robin Arnold <robin.arnold@ibm.com>

* issue #3437 fix usage of logicalResourceId and resourceId for remote index

Signed-off-by: Robin Arnold <robin.arnold@ibm.com>
@punktilious
issue #3437 inject instanceIdentifier when sending remote index payload
6c8d712 
Signed-off-by: Robin Arnold <robin.arnold@ibm.com>
@lmsurpre lmsurpre self-requested a review June 21, 2022 13:02
lmsurpre
lmsurpre approved these changes Jun 21, 2022
View reviewed changes
Copy link
Member

@lmsurpre lmsurpre left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lots of good stuff in here. I reviewed all the files and ran locally for bit with remote indexing. I didn't run any of the Citus stuff.

@punktilious
issue #3437 more schema design documentation
59e3c24 
Signed-off-by: Robin Arnold <robin.arnold@ibm.com>
@punktilious punktilious added ci-skip Skips the CI Build and removed ci-skip Skips the CI Build labels Jun 21, 2022
@lmsurpre lmsurpre merged commit b6043f3 into main Jun 21, 2022
@lmsurpre lmsurpre deleted the robin-citus-1 branch June 21, 2022 14:02
@lmsurpre lmsurpre added the reindex Resolution of issue will require a $reindex during upgrade label Aug 22, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@lmsurpre lmsurpre lmsurpre approved these changes

Assignees
No one assigned
Labels
new-module Flag for identifying PRs that introduce a new module (and therefor require manual work for release) reindex Resolution of issue will require a $reindex during upgrade
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@punktilious @lmsurpre