Move ncbi_api_key to secrets.

[anna-parker]

resolves #1844

preview URL: https://set-secret-ncbi.loculus.org/

Steps to seal

1. Switch to ~/.kube/config with our cluster credentials
2. Create a secret.yaml, where the key is in base64 (echo -n KEY | base64) - create secrets does this conversion for you-

kubectl create secret generic ingest-ncbi --from-literal=api-key=KEY --dry-run=client -o yaml > secret.yaml

3. Seal the secret using

kubeseal --format yaml   --scope=cluster-wide < secret.yaml

4. Add to kubernetes/loculus/values_preview_server.yaml.

Screenshot

PR Checklist

• Check still runs on local dev instance without api-key
• Check runs on argocd
• Seal secret using https://github.com/bitnami-labs/sealed-secrets/tree/main

[theosanderson]

You need to add a flag to the seal command so it can be unsealed in any namespace – I'll look up the flag when not afk

[anna-parker]

@theosanderson (absolutely no urgency) as you set this up for other secrets - which namespace is the sealed-secrets-controller in? (I'm also running these commands locally - which I hope is ok - I'm a bit uncertain how exactly kubeseal on my machine knows which public key the controller is using for sealing this secret... so I'm wondering if this isn't actually the right way - but the docs are a bit vague)

[anna-parker]

They talk about getting the public key from the cluster for encryption: https://github.com/bitnami-labs/sealed-secrets?tab=readme-ov-file#public-key--certificate - but don't we have different ones for each instance?

[theosanderson]

Yeah the docs are confusing on that. For our k3s cluster, all you need to do is:

• have a set up where kubectl is telling you things about OUR cluster, i.e. your ~/.kube/config contains credentials for our cluster and you have picked the right cluster
• Run this command kubeseal --format yaml --scope=cluster-wide < secret.yaml where secret.yaml is a local file with your secret
• That will output the sealed secret to the terminal

The kubeseal docs are confusing - you only need to specify the controller stuff when that is set to non-default values (for our Amazon cluster we have set that to non-default stuff so one does need to specify this)

[anna-parker]

Ah my ~/.kube/config currently points to my local k3d-testCluster - can we call tomorrow and you show me how to modify this to the k3d cluster?

[theosanderson]

Have you ever been able to access our k3s cluster? If not you need credentials which until now I think Chaoran has issued

[theosanderson]

Cornelius also wrote up some stuff about it: https://github.com/loculus-project/loculus/blob/main/kubernetes/README.md#setting-up-kubeconfig-locally-to-access-the-remote-cluster

[anna-parker]

I tried changing the secret.yaml but keep getting the same error:

Run ./deploy.py --verbose helm --branch 2357/merge --sha d786b3e --for-e2e
Error: INSTALLATION FAILED: unable to build kubernetes objects from release manifest: resource mapping not found for name: "ingest-ncbi" namespace: "" from "": no matches for kind "SealedSecret" in version "bitnami.com/v1alpha1"
ensure CRDs are installed first

[theosanderson]

Ah, sealed secrets only apply to stuff that happens on the cluster. I.e. they won't work for the E2E tests. Maybe put the secret in the preview specific yaml? (In the same directory as values.yaml)

[theosanderson]

Nice!