Is Talos suitable for use without Kubernetes

[kurtismullins]

I'm interested in provisioning machines with an immutable OS since I only need to run a few processes on each machine and the machines (aside from an attached volume/drive) can be ephemeral. I don't need a full Kubernetes environment. I also don't want to use Ansible or other configuration management utilities because the machine's state can sway from the intended config over time; it's much easier to just re-provision your machine with a new init script.

Is anyone using Talos for this purpose? Are there any downsides specific to Talos, especially compared to other immutable OS distributions, for doing this kind of thing?

Thanks!

[smira]

This is something Talos Linux can certainly do, but there are some missing pieces to make it fully Kubernetes-less.

I haven't tried that, but you could do a single controlplane node, disable kubelet node registration, skip controller-manager & scheduler (api-server will still be started), and use .machine.pods to launch static pods.

This way nothing can be scheduled on the machine using Kubernetes APIs, but you can manage your workloads modifying .machine.pods.

[rothgar]

Thanks for the suggestion Kurt. It's absolutely something that has been on our minds and an eventual long term goal of having an immutable, API driven, single purpose Linux distribution. We strongly believe that making Linux do 1 thing makes them more stable and maintainable long term.

But anything we build on top of Talos would be a bit of a hack. Talos will always stay a single-purpose Kubernetes distribution. We would create different single-purpose distributions for other use cases so we don't overly complicate the machine API and the distros can remain focused.

Could you let us know how you'd like to distribute your software to the nodes (e.g. containers, binaries)? Do you expect machines to be highly available or part of a cluster, not Kubernetes? Do you want centralized OS management (eg Omni) or is this for individual use that you would manage 1-by-1?

Right now we don't have any plans to build this but we're gathering data about use cases to see what we can do in the future.

[kurtismullins]

Talos will always stay a single-purpose Kubernetes distribution

Thanks for letting me know!

Could you let us know how you'd like to distribute your software to the nodes (e.g. containers, binaries)?

Primarily, containers. However, some binaries make sense to run outside of a container (think node-exporter, cadvisor, etc..) just due to the complex configuration required to grant access to everything they need.

Do you expect machines to be highly available or part of a cluster, not Kubernetes?

Both! In my current employer's case, we do not use Kubernetes because the traditional "deployment" model doesn't work for our software; it handles replication, internal routing, etc... itself. We would affectively just be running N replicas and using Kubernetes features to limit it to a single process per machine and attempting to limit Disk IO to the localhost. A lot of the functionality would be replicated. This can be a pain to properly configure and requires everyone on-call to understand maintaining Kubernetes systems on bare metal.

For personal purposes, I could see using it as a lightweight, immutable server OS with a nice API.

Do you want centralized OS management (eg Omni)

I have never used Omni but my preference would be more agnostic tooling such as Terraform and Ansible. Preferably just being able to configure everything initially with cloud-init/ignition/etc... would be a wonderful start.

is this for individual use that you would manage 1-by-1

Maybe a nice-to-have but definitely not my business case.