Standardize on container images instead of machine images

[0x2b3bfa0]

Follow-up of #127 (comment)

It would be nice to offer a single, consistent environment on every platform, and we can ship default container images as part of the machine images to avoid pull delays and costs.

This proposal assumes that:

• The user–provided code is intended to (or at least can) run on Linux.
• Users who have on–premises GPU farms are able to install Docker.

I'm inclined to think that those assumptions are pretty reasonable, and a good compromise between impact and effort on our side.

[0x2b3bfa0]

If future versions of CRIU support loading/restoring the internal state of CUDA devices, standardizing on containers could have the additional advantage of allowing us to perform live migrations between spot instances. The advantages versus data-based checkpoints aren't especially obvious, but it looks like the next cool technology. 😄 See also #176 (comment)

[0x2b3bfa0]

Blockers for containerized cml runner

From all the continuous integration systems we support,¹ GitHub Actions is the only that doesn't play nicely with containerized self-hosted runners:

• Support Runner inside of Docker Container actions/runner#406
• Provide runner as a Docker Image actions/runner#367
• runner --driver=github Docker-in-Docker cml#908 (comment)

Footnotes

1. Namely, GitHub Actions, GitLab CI/CD and Bitbucket Pipelines. ↩

[0x2b3bfa0]

Machine images offered by providers have lots of quirks and don't include any of the helper tools we need to offer a good user experience.

Custom images are the only alternative to provisioning instances on the fly, but forcing users to run tasks in a fixed environment could be unwise. Especially when it implies committing to build and maintain a stable and secure reference image.

Resposiveness-wise, the most appropriate solution would be using containers or lightweight virtual machines with user-specified images, including some default general purpose images with our custom machine images in order to reduce load times.

• Implement GPU passthrough functionality to provide hw-acceleration to inner-containers nestybox/sysbox#50
• https://github.com/kata-containers/documentation/blob/master/use-cases/Nvidia-GPU-passthrough-and-Kata.md
• [Devices] Offer support for hardware-accelerated inference in Firecracker firecracker-microvm/firecracker#1179
• https://blog.cloudkernels.net/posts/vaccel_v2/
• support for nvidia-docker GPU container sandboxing google/gvisor#14

Moved from the experimental XPD library.

[casperdcl]

do you mean allow resource "iterative_task" { image = "docker://..." }?

[0x2b3bfa0]

This issue predates the iterative_task resource, but yes.

[0x2b3bfa0]

allow resource "iterative_task" { image = "docker://..." }

🪓

terraform {
  required_providers {
    iterative = { source = "iterative/iterative" }
  }
}

provider "iterative" {}

resource "iterative_task" "example" {
  cloud   = "aws"
  image   = "nvidia"
  machine = "g4dn.xlarge"

  script = <<-END
    #!/usr/bin/env -S sh -c 'docker run --rm -iv "$(realpath "$0"):/file" alpine sh /file'
    cat /etc/alpine-release
  END
}