Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Backport 2.x] Implement multi tenancy in Flow Framework (#980) #1024

Merged
merged 1 commit into from
Jan 25, 2025

Conversation

dbwiddis
Copy link
Member

Backports d8a5a4b from #980

* Import SdkClient and inject it

Signed-off-by: Daniel Widdis <widdis@gmail.com>

* Pass sdkClient to IndicesHandler and EncryptorUtils classes

Signed-off-by: Daniel Widdis <widdis@gmail.com>

* Extract tenant id from REST header into RestAction

Signed-off-by: Daniel Widdis <widdis@gmail.com>

* Pass tenant id to transport actions in template

Signed-off-by: Daniel Widdis <widdis@gmail.com>

* Validate tenant id existence in workflow transport actions

Signed-off-by: Daniel Widdis <widdis@gmail.com>

* Pass SdkClient and tenant id to util used for access control checks

Signed-off-by: Daniel Widdis <widdis@gmail.com>

* Perform tenant id validation checks for workflow APIs

Signed-off-by: Daniel Widdis <widdis@gmail.com>

* Migrate Update workflow get action to SdkCleint

Signed-off-by: Daniel Widdis <widdis@gmail.com>

* Pass tenantId to IndicesHandler and use in EncryptorUtils

Signed-off-by: Daniel Widdis <widdis@gmail.com>

* Migrate EncryptorUtils getting master key from index

Signed-off-by: Daniel Widdis <widdis@gmail.com>

* Refactor fetching master key to permit reuse

Signed-off-by: Daniel Widdis <widdis@gmail.com>

* Refactor initializeMasterKey to use common code

Signed-off-by: Daniel Widdis <widdis@gmail.com>

* Migrate indexing new key to config

Signed-off-by: Daniel Widdis <widdis@gmail.com>

* Migrate template indexing to sdkClient

Signed-off-by: Daniel Widdis <widdis@gmail.com>

* Migrate template deletion to sdkClient

Signed-off-by: Daniel Widdis <widdis@gmail.com>

* Migrate get template to sdkClient

Signed-off-by: Daniel Widdis <widdis@gmail.com>

* Migrate provision template to sdkClient

Signed-off-by: Daniel Widdis <widdis@gmail.com>

* Migrate max workflow search to sdkClient

Signed-off-by: Daniel Widdis <widdis@gmail.com>

* Add tenantId to GetWorkflowStateRequest

Signed-off-by: Daniel Widdis <widdis@gmail.com>

* Migrate GetWorkflowStateRequest to multitenant client

Signed-off-by: Daniel Widdis <widdis@gmail.com>

* Migrate getProvisioningProgress to avoid repetition

Signed-off-by: Daniel Widdis <widdis@gmail.com>

* Migrate canDeleteWorkflowStateDoc to avoid repetition

Signed-off-by: Daniel Widdis <widdis@gmail.com>

* Migrate initial state document creation to metadata client

Signed-off-by: Daniel Widdis <widdis@gmail.com>

* Migrate state document deletion to metadata client

Signed-off-by: Daniel Widdis <widdis@gmail.com>

* Add Tenant aware Rest Tests for Workflows

Signed-off-by: Daniel Widdis <widdis@gmail.com>

* Fix javadocs

Signed-off-by: Daniel Widdis <widdis@gmail.com>

* Add publishToMavenLocal for more CI

Signed-off-by: Daniel Widdis <widdis@gmail.com>

* Fix some CI

Signed-off-by: Daniel Widdis <widdis@gmail.com>

* Enable tenant aware search

Signed-off-by: Daniel Widdis <widdis@gmail.com>

* Refactor state index update method using multitenant client

Signed-off-by: Daniel Widdis <widdis@gmail.com>

* Get metadata client artifacts from Maven Snapshot

Signed-off-by: Daniel Widdis <widdis@gmail.com>

* Update tests for new update async code

Signed-off-by: Daniel Widdis <widdis@gmail.com>

* Switch SdkClient to use default generic thread executor

Signed-off-by: Daniel Widdis <widdis@gmail.com>

* Migrate last updates to sdkClient

Signed-off-by: Daniel Widdis <widdis@gmail.com>

* Revert (most) changes to unit tests based on async client changes

Signed-off-by: Daniel Widdis <widdis@gmail.com>

* Pass tenant id when updating state during provisioning

Signed-off-by: Daniel Widdis <widdis@gmail.com>

* Integrate tenantId with synchronous provisioning

Signed-off-by: Daniel Widdis <widdis@gmail.com>

* Fix failing integ tests after rebase, code review updates

Signed-off-by: Daniel Widdis <widdis@gmail.com>

* Replace fakeTenantId placeholders with actual tenant id

Signed-off-by: Daniel Widdis <widdis@gmail.com>

* Use version catalog for commons-lang3 and httpcore dependencies

Signed-off-by: Daniel Widdis <widdis@gmail.com>

* Exclude transitive httpclient dependency from metadata and rest client

Signed-off-by: Daniel Widdis <widdis@gmail.com>

* Fix more test errors and tweak dependencies

Signed-off-by: Daniel Widdis <widdis@gmail.com>

* More code review comments and refactoring

Signed-off-by: Daniel Widdis <widdis@gmail.com>

---------

Signed-off-by: Daniel Widdis <widdis@gmail.com>
Copy link

codecov bot commented Jan 25, 2025

Codecov Report

Attention: Patch coverage is 62.74256% with 288 lines in your changes missing coverage. Please review.

Project coverage is 75.52%. Comparing base (c9442f4) to head (ff500da).
Report is 1 commits behind head on 2.x.

Files with missing lines Patch % Lines
...framework/indices/FlowFrameworkIndicesHandler.java 65.07% 82 Missing and 6 partials ⚠️
...ework/transport/CreateWorkflowTransportAction.java 66.05% 28 Missing and 9 partials ⚠️
.../opensearch/flowframework/util/EncryptorUtils.java 63.63% 26 Missing and 10 partials ⚠️
.../transport/ReprovisionWorkflowTransportAction.java 20.00% 14 Missing and 2 partials ⚠️
...ework/transport/DeleteWorkflowTransportAction.java 55.17% 10 Missing and 3 partials ⚠️
...flowframework/transport/handler/SearchHandler.java 45.83% 13 Missing ⚠️
.../org/opensearch/flowframework/util/ParseUtils.java 43.47% 9 Missing and 4 partials ⚠️
...h/flowframework/rest/RestCreateWorkflowAction.java 60.71% 6 Missing and 5 partials ⚠️
...rk/transport/ProvisionWorkflowTransportAction.java 47.05% 7 Missing and 2 partials ⚠️
.../opensearch/flowframework/FlowFrameworkPlugin.java 50.00% 7 Missing and 1 partial ⚠️
... and 13 more
Additional details and impacted files
@@             Coverage Diff              @@
##                2.x    #1024      +/-   ##
============================================
- Coverage     76.41%   75.52%   -0.89%     
- Complexity     1010     1050      +40     
============================================
  Files           100      101       +1     
  Lines          4884     5214     +330     
  Branches        455      498      +43     
============================================
+ Hits           3732     3938     +206     
- Misses          955     1038      +83     
- Partials        197      238      +41     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@dbwiddis dbwiddis merged commit 82bfbc3 into opensearch-project:2.x Jan 25, 2025
33 of 34 checks passed
@dbwiddis dbwiddis deleted the backport-980-to-2.x branch January 25, 2025 03:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants