Added Profile API

[Merey1508]

Description

These three methods all retrieve different types of profiles from an OpenSearch cluster using the ML Commons API. Here's a brief description of each method:

def get_profile(self, node_ids: List[str] = [], return_all_tasks: bool = True, return_all_models: bool = True) -> object:

This method retrieves a profile from an OpenSearch cluster for one or more specified node IDs.

def get_profile_models(self, model_ids: List[str] = []) -> object:

This method retrieves a profile from an OpenSearch cluster for one or more specified machine learning models.

def get_profile_tasks(self, task_ids: List[str] = []) -> object:

This method retrieves a profile from an OpenSearch cluster for one or more specified machine learning tasks.

All three methods are similar in structure and use the same API endpoint to retrieve different types of profiles based on the parameters passed to them.

Issues Resolved

Link to issue: #104

Check List

• New functionality includes testing.
  • All tests pass
• New functionality has been documented.
  • New functionality has javadoc added
• Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

[dhrubo-os]

1. We need to add test for each methods to make sure the functionality.
2. We need to add documentation for each method in more details including describing parameters. Check other methods please.
3. Linting is failing. check here

[Merey1508]

• We need to add test for each methods to make sure the functionality.
• We need to add documentation for each method in more details including describing parameters. Check other methods please.
• Linting is failing. check here

Thanks for your comment. I fixed linting. Could you check the documentation? If I'm not mistaken, I added the documentation within code by shortly describing each method and their parameters.

[dhrubo-os]

Can we try to merge all these 3 functions into one function?
It can be hard for customers to remember all these methods.

Can you please try to combine all these 3 functionalities into 1 function?

[Merey1508]

Ok, I will try. Thank you for bringing this to my attention.

[dhrubo-os]

we need to write corresponding test to make sure this is working or not.

[dhrubo-os]

Let's make this more robust so that we can also support scenarios like here

1. GET /_plugins/_ml/profile/models --> If we see only one model_id is given nothing else, we can invoke this api. Example: GET /_plugins/_ml/profile/models/model_id
2. GET /_plugins/_ml/profile/tasks --> if we see only one task_id is given nothing else, we can invoke this api.
   Example: GET /_plugins/_ml/profile/tasks/task_id
3. Otherwise invoke the base API. Example:

GET /_plugins/_ml/profile
{
  "node_ids": ["KzONM8c8T4Od-NoUANQNGg"],
  "return_all_tasks": true,
  "return_all_models": true
}

Please let me know if you have any confusion.

Also don't forget to write test. Let me know if you have any question regarding writing test.

[Nurlanprog]

1. Endpoint: GET /_plugins/_ml/profile/models/model_id works fine.
2. Endpoint: GET /_plugins/_ml/profile/tasks/task_id always returns empty json. So it was deleted. Instead for searching model tasks used API_BODY.
3. In testing function for traversing over dictionary was added because model_id and task_id in returned json is 4 levels deeper.

[rawwar]

1. GET /_plugins/_ml/profile/models --> If we see only one model_id is given nothing else, we can invoke this api. Example: GET /_plugins/_ml/profile/models/model_id
2. GET /_plugins/_ml/profile/tasks --> if we see only one task_id is given nothing else, we can invoke this api.
   Example: GET /_plugins/_ml/profile/tasks/task_id
3. Otherwise invoke the base API. Example:

@dhrubo-os :
For 1, If we implement that, and then user wants all model profile data, its going to cause confusion. Example: User wants all tasks data and so, he doesn't pass model_id nor task_id . How are we going to conclude what user wanted here?

I think, a rather simple interface is to give users to call the respective api's separately. we should have three separate methods each returning exactly what user is requesting. Accepting model_id or task_id can be optional. I propose the following methods

1. get_profiles(request_body)
2. get_model_profile(model_id=None, request_body=None)
3. get_model_tasks(task_id=None, request_body=None)

[dhrubo-os]

@Merey1508 is there any update on writing unit test? Please let me know if you need any help, thanks.

[codecov-commenter]

Codecov Report

Merging #119 (914083c) into main (7ff5113) will decrease coverage by 0.23%.
The diff coverage is 10.00%.

📣 This organization is not using Codecov’s GitHub App Integration. We recommend you install it so Codecov can continue to function properly for your repositories. Learn more

@@            Coverage Diff             @@
##             main     #119      +/-   ##
==========================================
- Coverage   89.97%   89.74%   -0.23%     
==========================================
  Files          29       29              
  Lines        3491     3501      +10     
==========================================
+ Hits         3141     3142       +1     
- Misses        350      359       +9     

Impacted Files	Coverage Δ
opensearch_py_ml/ml_commons/ml_commons_client.py	79.54% <10.00%> (-8.92%)	⬇️

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

[dhrubo-os]

Can we try to add a comment to describe what exactly are we doing here?

[dhrubo-os]

closing this pull request as this functionality is already done.