Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Documentation] Support ML node role #672

Closed
brijos opened this issue Jun 14, 2022 · 3 comments · Fixed by #787
Closed

[Documentation] Support ML node role #672

brijos opened this issue Jun 14, 2022 · 3 comments · Fixed by #787
Assignees
Labels
Closed - Complete Issue: Work is done and associated PRs closed v2.1.0
Milestone

Comments

@brijos
Copy link

brijos commented Jun 14, 2022

Is your feature request related to a problem?
ML model generally consuming more resources, especially for training process. The community wants to support bigger ML models which might require more resources and special hardware like GPU.

As OpenSearch doesn’t support ML node, we dispatch ML task to data node only. That means if user wants to train a large model, they need to scale up all data nodes which can be costly.

What solution would you like?
Support a dedicated ML node, users don’t need to scale up their data node at all. Instead just configure a new ML node (with different settings, more powerful instance type) and add it to cluster via the YAML file (requires a cluster restart). By doing so, users can separate resource usage better by running ML task on dedicated node which can reduce impact to other critical tasks like search/ingestion.

What alternatives have you considered?
opensearch-project/OpenSearch#2877

Do you have any additional context?
opensearch-project/OpenSearch#2877

Meta Reference - Meta - Support ML node role

@brijos brijos added enhancement New feature or request untriaged v2.1.0 labels Jun 14, 2022
@Naarcha-AWS
Copy link
Collaborator

@brijos: Is this issue implying that we might have ML node support by 2.1.0?

@Naarcha-AWS Naarcha-AWS self-assigned this Jun 16, 2022
@Naarcha-AWS Naarcha-AWS removed untriaged enhancement New feature or request labels Jun 16, 2022
@Naarcha-AWS Naarcha-AWS added this to the v2.1.0 milestone Jun 16, 2022
@Naarcha-AWS Naarcha-AWS assigned alicejw1 and unassigned Naarcha-AWS Jun 21, 2022
@Naarcha-AWS
Copy link
Collaborator

Need to figure out if this is still relevant to 2.1.

@Naarcha-AWS Naarcha-AWS assigned Naarcha-AWS and unassigned alicejw1 Jun 23, 2022
@Naarcha-AWS Naarcha-AWS added v2.2.0 and removed v2.1.0 labels Jun 23, 2022
@Naarcha-AWS Naarcha-AWS modified the milestones: v2.1.0, v2.2 Jun 23, 2022
@brijos brijos added v2.1.0 and removed v2.2.0 labels Jun 27, 2022
@Naarcha-AWS Naarcha-AWS modified the milestones: v2.2, v2.1.0 Jun 30, 2022
@ylwu-amzn
Copy link
Contributor

@brijos: Is this issue implying that we might have ML node support by 2.1.0?

Yes, we are going to release this in 2.1.0.

@Naarcha-AWS Naarcha-AWS added the 2 - In progress Issue/PR: The issue or PR is in progress. label Jul 6, 2022
@hdhalter hdhalter added Closed - Complete Issue: Work is done and associated PRs closed and removed 2 - In progress Issue/PR: The issue or PR is in progress. labels Aug 1, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Closed - Complete Issue: Work is done and associated PRs closed v2.1.0
Projects
None yet
Development

Successfully merging a pull request may close this issue.

5 participants