model access control changes rebased to 2.x #902

rbhavna · 2023-05-19T06:08:15Z

Description

model access control changes rebased to 2.x

Issues Resolved

Check List

New functionality includes testing.
- All tests pass
New functionality has been documented.
- New functionality has javadoc added
Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

…roject#837) * rename model meta/chunk API (opensearch-project#827) Signed-off-by: Yaliang Wu <ylwu@amazon.com> * Change the ziputil dependency to fix a potential security concern (opensearch-project#824) Signed-off-by: Yaliang Wu <ylwu@amazon.com> * add text docs ML input (opensearch-project#830) Signed-off-by: Yaliang Wu <ylwu@amazon.com> --------- Signed-off-by: Yaliang Wu <ylwu@amazon.com> Co-authored-by: Sicheng Song <sicheng.song@outlook.com>

* add model group Signed-off-by: Yaliang Wu <ylwu@amazon.com> * rename create model group as register model group Signed-off-by: Yaliang Wu <ylwu@amazon.com> * fix class name in build.gradle Signed-off-by: Yaliang Wu <ylwu@amazon.com> * remove unused code; stash thread context Signed-off-by: Yaliang Wu <ylwu@amazon.com> * exclude class for low coverage Signed-off-by: Yaliang Wu <ylwu@amazon.com> --------- Signed-off-by: Yaliang Wu <ylwu@amazon.com>

…e/deploy/predict model Signed-off-by: Bhavana Ramaram <rbhavna@amazon.com>

Signed-off-by: Bhavana Ramaram <rbhavna@amazon.com>

codecov · 2023-05-19T06:19:15Z

Codecov Report

Merging #902 (91ca331) into model-access-control-dev (134e065) will decrease coverage by 11.80%.
The diff coverage is 14.73%.

❗ Current head 91ca331 differs from pull request most recent head 55295ee. Consider uploading reports for the commit 55295ee to get more accurate results

@@                       Coverage Diff                       @@
##             model-access-control-dev     #902       +/-   ##
===============================================================
- Coverage                       84.18%   72.39%   -11.80%     
+ Complexity                       1764     1638      -126     
===============================================================
  Files                             139      147        +8     
  Lines                            6780     7281      +501     
  Branches                          676      711       +35     
===============================================================
- Hits                             5708     5271      -437     
- Misses                            774     1729      +955     
+ Partials                          298      281       -17

Flag	Coverage Δ
ml-commons	`72.39% <14.73%> (-11.80%)`	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
...rg/opensearch/ml/client/MachineLearningClient.java	`100.00% <ø> (ø)`
...pensearch/ml/client/MachineLearningNodeClient.java	`96.07% <ø> (ø)`
...ction/deploy/TransportDeployModelOnNodeAction.java	`16.66% <0.00%> (-80.74%)`	⬇️
...va/org/opensearch/ml/indices/MLIndicesHandler.java	`80.55% <0.00%> (+10.55%)`	⬆️
.../opensearch/ml/rest/RestMLUndeployModelAction.java	`35.71% <0.00%> (-54.99%)`	⬇️
...in/java/org/opensearch/ml/utils/SecurityUtils.java	`0.00% <0.00%> (ø)`
...h/ml/action/deploy/TransportDeployModelAction.java	`14.70% <2.43%> (-69.85%)`	⬇️
.../action/register/TransportRegisterModelAction.java	`21.42% <3.38%> (-77.21%)`	⬇️
...h/ml/action/models/DeleteModelTransportAction.java	`9.41% <10.00%> (-71.42%)`	⬇️
...n/java/org/opensearch/ml/model/MLModelManager.java	`66.91% <12.64%> (-10.38%)`	⬇️
... and 16 more

... and 14 files with indirect coverage changes

Signed-off-by: Bhavana Ramaram <rbhavna@amazon.com>

ylwu-amzn · 2023-05-22T22:49:12Z

common/src/main/java/org/opensearch/ml/common/CommonValue.java

@@ -43,6 +44,48 @@ public class CommonValue {
            + "          \"custom_attribute_names\": {\"type\":\"text\", \"fields\":{\"keyword\":{\"type\":\"keyword\"}}}\n"
            + "        }\n"
            + "      }\n";
+    public static final String ML_MODEL_GROUP_INDEX_MAPPING = "{\n" +
+            "  \"_meta\": {\n" +
+            "    \"schema_version\": "+ML_MODEL_GROUP_INDEX_SCHEMA_VERSION+"\n" +


Minor : add whitespace before and after +

ylwu-amzn · 2023-05-23T17:38:19Z

common/src/main/java/org/opensearch/ml/common/CommonValue.java

@@ -43,6 +44,48 @@ public class CommonValue {
            + "          \"custom_attribute_names\": {\"type\":\"text\", \"fields\":{\"keyword\":{\"type\":\"keyword\"}}}\n"
            + "        }\n"
            + "      }\n";
+    public static final String ML_MODEL_GROUP_INDEX_MAPPING = "{\n" +


@rbhavna I don't see tags field in ML_MODEL_GROUP_INDEX_MAPPING. As tags will be key value pairs, suggest take a look at flat object

* Backport changes to model-access-control feature branch (opensearch-project#837) * rename model meta/chunk API (opensearch-project#827) Signed-off-by: Yaliang Wu <ylwu@amazon.com> * Change the ziputil dependency to fix a potential security concern (opensearch-project#824) Signed-off-by: Yaliang Wu <ylwu@amazon.com> * add text docs ML input (opensearch-project#830) Signed-off-by: Yaliang Wu <ylwu@amazon.com> --------- Signed-off-by: Yaliang Wu <ylwu@amazon.com> Co-authored-by: Sicheng Song <sicheng.song@outlook.com> * add model group (opensearch-project#840) * add model group Signed-off-by: Yaliang Wu <ylwu@amazon.com> * rename create model group as register model group Signed-off-by: Yaliang Wu <ylwu@amazon.com> * fix class name in build.gradle Signed-off-by: Yaliang Wu <ylwu@amazon.com> * remove unused code; stash thread context Signed-off-by: Yaliang Wu <ylwu@amazon.com> * exclude class for low coverage Signed-off-by: Yaliang Wu <ylwu@amazon.com> --------- Signed-off-by: Yaliang Wu <ylwu@amazon.com> * access validation for register/update model-group, register/get/delete/deploy/predict model Signed-off-by: Bhavana Ramaram <rbhavna@amazon.com> * changes to security utils and register/search model group Signed-off-by: Bhavana Ramaram <rbhavna@amazon.com> * update last_updated_time in model group when new version added Signed-off-by: Bhavana Ramaram <rbhavna@amazon.com> * fix undeploy model action Signed-off-by: Bhavana Ramaram <rbhavna@amazon.com> * fix format violations Signed-off-by: Bhavana Ramaram <rbhavna@amazon.com> * rebased to 2.x and prediction/search API fix Signed-off-by: Bhavana Ramaram <rbhavna@amazon.com> * fix rebase conflicts Signed-off-by: Bhavana Ramaram <rbhavna@amazon.com> * new delete model group API Signed-off-by: Bhavana Ramaram <rbhavna@amazon.com> * fix formal violations Signed-off-by: Bhavana Ramaram <rbhavna@amazon.com> --------- Signed-off-by: Yaliang Wu <ylwu@amazon.com> Signed-off-by: Bhavana Ramaram <rbhavna@amazon.com> Co-authored-by: Yaliang Wu <ylwu@amazon.com> Co-authored-by: Sicheng Song <sicheng.song@outlook.com>

* Backport changes to model-access-control feature branch (opensearch-project#837) * rename model meta/chunk API (opensearch-project#827) Signed-off-by: Yaliang Wu <ylwu@amazon.com> * Change the ziputil dependency to fix a potential security concern (opensearch-project#824) Signed-off-by: Yaliang Wu <ylwu@amazon.com> * add text docs ML input (opensearch-project#830) Signed-off-by: Yaliang Wu <ylwu@amazon.com> --------- Signed-off-by: Yaliang Wu <ylwu@amazon.com> Co-authored-by: Sicheng Song <sicheng.song@outlook.com> * add model group (opensearch-project#840) * add model group Signed-off-by: Yaliang Wu <ylwu@amazon.com> * rename create model group as register model group Signed-off-by: Yaliang Wu <ylwu@amazon.com> * fix class name in build.gradle Signed-off-by: Yaliang Wu <ylwu@amazon.com> * remove unused code; stash thread context Signed-off-by: Yaliang Wu <ylwu@amazon.com> * exclude class for low coverage Signed-off-by: Yaliang Wu <ylwu@amazon.com> --------- Signed-off-by: Yaliang Wu <ylwu@amazon.com> * access validation for register/update model-group, register/get/delete/deploy/predict model Signed-off-by: Bhavana Ramaram <rbhavna@amazon.com> * changes to security utils and register/search model group Signed-off-by: Bhavana Ramaram <rbhavna@amazon.com> * update last_updated_time in model group when new version added Signed-off-by: Bhavana Ramaram <rbhavna@amazon.com> * fix undeploy model action Signed-off-by: Bhavana Ramaram <rbhavna@amazon.com> * fix format violations Signed-off-by: Bhavana Ramaram <rbhavna@amazon.com> * rebased to 2.x and prediction/search API fix Signed-off-by: Bhavana Ramaram <rbhavna@amazon.com> * fix rebase conflicts Signed-off-by: Bhavana Ramaram <rbhavna@amazon.com> * new delete model group API Signed-off-by: Bhavana Ramaram <rbhavna@amazon.com> * fix formal violations Signed-off-by: Bhavana Ramaram <rbhavna@amazon.com> --------- Signed-off-by: Yaliang Wu <ylwu@amazon.com> Signed-off-by: Bhavana Ramaram <rbhavna@amazon.com> Co-authored-by: Yaliang Wu <ylwu@amazon.com> Co-authored-by: Sicheng Song <sicheng.song@outlook.com> Signed-off-by: Sicheng Song <sicheng.song@outlook.com>

ylwu-amzn and others added 8 commits May 17, 2023 02:09

access validation for register/update model-group, register/get/delet…

f6c8780

…e/deploy/predict model Signed-off-by: Bhavana Ramaram <rbhavna@amazon.com>

changes to security utils and register/search model group

3c99d7c

Signed-off-by: Bhavana Ramaram <rbhavna@amazon.com>

update last_updated_time in model group when new version added

e41cc1d

Signed-off-by: Bhavana Ramaram <rbhavna@amazon.com>

fix undeploy model action

112934d

Signed-off-by: Bhavana Ramaram <rbhavna@amazon.com>

fix format violations

0aad57e

Signed-off-by: Bhavana Ramaram <rbhavna@amazon.com>

rebased to 2.x and prediction/search API fix

91ca331

Signed-off-by: Bhavana Ramaram <rbhavna@amazon.com>

fix rebase conflicts

29a0241

Signed-off-by: Bhavana Ramaram <rbhavna@amazon.com>

zane-neo self-requested a review May 22, 2023 06:28

zane-neo previously approved these changes May 22, 2023

View reviewed changes

new delete model group API

e54e399

Signed-off-by: Bhavana Ramaram <rbhavna@amazon.com>

rbhavna dismissed zane-neo’s stale review via e54e399 May 22, 2023 12:30

fix formal violations

55295ee

Signed-off-by: Bhavana Ramaram <rbhavna@amazon.com>

ylwu-amzn reviewed May 22, 2023

View reviewed changes

zane-neo approved these changes May 23, 2023

View reviewed changes

ylwu-amzn approved these changes May 23, 2023

View reviewed changes

zane-neo merged commit e49227b into opensearch-project:model-access-control-dev May 23, 2023

ylwu-amzn reviewed May 23, 2023

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

model access control changes rebased to 2.x #902

model access control changes rebased to 2.x #902

rbhavna commented May 19, 2023

codecov bot commented May 19, 2023 •

edited

Loading

ylwu-amzn May 22, 2023

ylwu-amzn May 23, 2023

model access control changes rebased to 2.x #902

model access control changes rebased to 2.x #902

Conversation

rbhavna commented May 19, 2023

Description

Issues Resolved

Check List

codecov bot commented May 19, 2023 • edited Loading

Codecov Report

ylwu-amzn May 22, 2023

Choose a reason for hiding this comment

ylwu-amzn May 23, 2023

Choose a reason for hiding this comment

codecov bot commented May 19, 2023 •

edited

Loading