TOB-FUEL-24: Secrets are passed over CLI and environment variables

[xgreenx]

Description

The fuel-core node expects the P2P key to be passed through the CLI with the argument --keypair. As a result, an attacker can steal the key by watching the list of processes.

Figure 24.1: Helm chart which passes a secret via the CLI. (fuel-core/deployment/charts/templates/fuel-core-deploy.yaml#211–214)

{{- if .Values.app.p2p_key }}
  - "--keypair"
  - "{{ .Values.app.p2p_key }}"
{{- end }}

Similarly, the consensus key is passed via an environment variable called CONSENSUS_KEY_SECRET. Keeping secrets in environment variables is a well-known antipattern. Environment variables are commonly captured by all manner of debugging and logging information, can be accessed from procfs, and are passed down to all child processes. Environment variables are in no way an acceptable substitute for a real secret-management system such as Vault or Keywhiz.

Figure 24.2: Helm chart which creates an environment variable containing a secret. (fuel-core/deployment/charts/templates/fuel-core-deploy.yaml#288–292)

- name: CONSENSUS_KEY_SECRET
  valueFrom:
    secretKeyRef:
      name: fuel-core-secret
      key: CONSENSUS_KEY_SECRET

Exploit Scenario

An attacker exploits another process on the same host as fuel-core and uses it to get the P2P key from the list of processes or read the consensus secret from procfs.

Recommendations

Short term, switch to a purpose-built secret-management system such as Vault or Keywhiz. Depending on how the fuel-core node is deployed there might be the need for a “secret zero”, which is required to unlock the vault. Either require the manual interaction of a human operator to enter this secret when the service is restarted, or store it on disk. If stored on disk, then protect it using the file permission 0o600 and run the fuel-core node as a separate user.
Long term, ensure that no secrets are stored in either code or environment variables, and use the secret manager instead.

[xgreenx]

@Voxelot @rfuelsh Is that a problem with the operator approach?

[xgreenx]

The operator uses k8s secrets for sensitive data. We mark the issue as acknowledged. We'd like to do this at some point, but it is a lower priority to solve as there could be many other issues besides key management if a pod is compromised.

We don't close this issue to continue tracking it in our backlog.

[xgreenx]

The fuel-core is integrated with AWS, plus we provide several options how to pass secrets. So the issue is resolve for now