How to configure for GPUs using Opta GCP directions?

[tszumowski]

I'm interested in trying out Flyte on GCP with tasks that vary in CPU, memory, and GPU usage. I noticed this tutorial provides a deployment from Opta.

However, I saw in the yaml here it provides an autoscaling (3-6 node) e2-medium node pool for the cluster.

  - type: k8s-cluster
    min_nodes: 3
    max_nodes: 6
    node_instance_type: e2-medium

How does one modify this configuration to provide the ability to use GPU enabled machines from a separate node pool? I saw these taint/tolerations directions, but was a little confused on how to tie it all together.

[brucearctor]

@tszumowski -- I am almost certain that is a typo. Suggesting to try by removing this line: https://github.com/flyteorg/flyte/blob/master/opta/gcp/env.yaml#L15

[brucearctor]

From: https://docs.opta.dev/reference/google/modules/gcp-gke/

the default instance size is: 'n2-highcpu-4'. Otherwise, you could follow those docs to change to the type/size that makes sense to you.

[tszumowski]

@brucearctor thank you for the quick reply. That is helpful for setting the base pool configuration. Though I'm more interested in having an as-needed GPU pool available and assignable in flyte such that it's effectively a one-liner for the Data Science User (after configured). I saw the gcp-nodepool option, but wasn't sure how to tie this together with Flyte config to provide a good DS user experience. Thoughts?

[brucearctor]

Suggesting to write a GitHub Issue ( In this case, I imagine a 'Documentation bug/update request' ), outlining what you're seeking. If this is something the team thinks others are interested in - then ideally some additional docs [ assuming don't already exist ] would get created and/or perhaps a feature to help facilitate what you're seeking could get created.

[brucearctor]

We are in the process of updating our deployment guides/opta to ensure that they are more straightforward to follow.

[tszumowski]

@brucearctor 👍 just saw this after writing my other reply. I look forward to the guides.

[kumare3]

hmm let me ping some folks who actually help with the gcp opta files to help with this. My knowledege on this is limited. cc @pmahindrakar-oss

[pmahindrakar-oss]

Can you try this in your opta file by defining a new node group

modules:
......

  - type: aws-nodegroup
    name: gpunodes
    use_gpu: true
    min_nodes: 1
    max_nodes: 5 # max gpu is 8 in #356633062068
    node_disk_size: 500
    node_instance_type: p2.xlarge

Following is doc from opta for it https://docs.opta.dev/reference/aws/modules/aws-nodegroup/
Opta support have a slack channel here if you face any additional issues https://opta-group.slack.com/archives/C0240NBKUMT

[brucearctor]

Do you want - type: aws-nodegroup in a nodegroup for GCP?

[pmahindrakar-oss]

Haven't tried this on our end but i couldn't find anything specific for gcp in there docs other than these https://docs.opta.dev/reference/google/modules/gcp-nodepool/
hence the above is just example format to be used . the gpu instance type and actual opta module to be used if not the above one i would suggest to be checked with opta team directly.

[wild-endeavor]

https://app.slack.com/team/U03R43ZK55Y
OP:
Newcomer question. I was wondering if anyone had tips on how to configure flyte to access GPUs. I see this document:
https://docs.flyte.org/projects/cookbook/en/latest/auto/deployment/configure_use_gpus.html#sphx-glr-auto-deployment-configure-use-gpus-py

But I am curious how to apply that configuration shown:

plugins:
  k8s:
    resource-tolerations:
      - [nvidia.com/gpu:](http://nvidia.com/gpu:)
        - key: "key1"
          operator: "Equal"
          value: "value1"
          effect: "NoSchedule"

I saw this thread which mentioned updating the propellor configmap. Is that the way to go? Or is there a way to update plugins via the flyte config?

Thanks! (edited)

Me:
how is your flyte cluster deployed?

OP:
I used the Opta GCP deployment guide and adjusted a bit after the deployment.

Me:
i think the answer will depend on how you deploy it. if you’re using the flyte helm chart for instance, you’ll want it so that it ends up here

OP:
I was able to get it running and assigned to my GLU node pool by manually editing the propeller configmap and doing a rolling restart. Just wondering if there was a best practice to how to edit those other than me manually editing 🙂

The Opta deployment uses the helm chart. So sounds like that is the way to go for me. Thanks!