cli/cmd: implement kubeconfig fallback to Terraform state

[invidian]

This PR includes commits from #631, please review this one first and don't leave commit comments for this PR here.

TODO:

• Running lokoctl health is now slower, as on every run, it initializes the Terraform and reads the output. Should we try to read file from assets dir first and then fallback to Terraform output? Maybe we should cache somehow the kubeconfig file content?
• Update and add new tests

Closes #608

Signed-off-by: Mateusz Gozdek mateusz@kinvolk.io

[invidian]

Running lokoctl health is now slower, as on every run, it initializes the Terraform and reads the output. Should we try to read file from assets dir first and then fallback to Terraform output? Maybe we should cache somehow the kubeconfig file content?

It seems caching is more complex than I thought, as the optimal sequence of execution is against our abstraction. What we need:

• Try loading platform configuration
• Check if kubeconfig file exists in platform assets directory (if it does, great, we're done
• initialize Terraform
• read Terraform output

And what we have right now:

• Try loading platform configuration
• initialize Terraform
• read Terraform output

[johananl]

This PR includes commits from #631, please review this one first and don't leave commit comments for this PR here.

What do you mean by "this one" and "this PR"? #631 or #701? Should #701 be reviewed?

[invidian]

What do you mean by "this one" and "this PR"? #631 or #701? Should #701 be reviewed?

It's not relevant anymore, as #631 has been merged already :) And yes, I requested reviews on this PR, so it should be reviewed.

[invidian]

Rebased to resolve conflicts.

[knrt10]

Looks great.

[surajssd]

This failed as we want:

$ lokoctl component apply
WARN[0000] Kubeconfig file not found in assets directory, pulling kubeconfig from Terraform state, this might be slow. Run 'lokoctl cluster apply' to fix it.  args="[]" command="lokoctl component apply"
INFO[0004] Executing step "initialize Terraform"         phase=infrastructure

You can find the logs in "assets/terraform/logs/299669.log"
FATA[0005] Error in finding kubeconfig file: reading kubeconfig file content from Terraform state: failed getting Terraform output for key "kubeconfig": exit status 1  args="[]" command="lokoctl component apply"

$ cat assets/terraform/logs/299669.log
Initializing modules...

Initializing the backend...

Initializing provider plugins...

Terraform has been successfully initialized!

You may now begin working with Terraform. Try running "terraform plan" to see
any changes that are required for your infrastructure. All Terraform commands
should now work.

If you ever set or change modules or backend configuration for Terraform,
rerun this command to reinitialize your working directory. If you forget, other
commands will detect it and remind you to do so if necessary.

Can we give more informational message to the user?

[invidian]

Can we give more informational message to the user?

Hmm, should we replace top-level message (Error in finding kubeconfig file: reading kubeconfig file content from Terraform state: failed getting Terraform output for key "kubeconfig": exit status 1) entirely then?

Something like Finding kubeconfig file failed. Did you run "lokoctl cluster apply" ? ?

[surajssd]

@invidian that should do for now :-) It is better than the terraform one.

[invidian]

Maybe we can keep existing message in debug verbosity?

[surajssd]

Sounds ok @invidian

[invidian]

Sounds ok @invidian

😕 I had to add --debug flag, so debug messages can be actually displayed. I could just set it without the flag too, but I think it's a reasonable addition. And I would really prefer not to hide the error. Alternative would be to print exact error after friendly error message.

[knrt10]

Can you please update the docs again, it's failing because of the init function change

[invidian]

Can you please update the docs again, it's failing because of the init function change

🤦 done 😄

[knrt10]

Rest lgtm

[knrt10]

+1

[surajssd]

This failed as follows:

$ lokoctl health --kubeconfig-file $HOME/.kube/config
ERRO[0000] <nil>: no platform configured;                args="[]" command="lokoctl health"
FATA[0000] Suitable kubeconfig file not found. Did you run 'lokoctl cluster apply' ?  args="[]" command="lokoctl health"

$ lokoctl version
v0.3.0-207-gf7f5eded9

$ ll
.rw-rw-r--@ 35 surajd  3 Sep 12:45 foobar.lokocfg

$ cat foobar.lokocfg 
component "prometheus-operator" {}

lokoctl still tries to get the file from state, even after explicit mention.

[invidian]

lokoctl still tries to get the file from state, even after explicit mention.

This is as intended. health command should be only run on L8e clusters as far as I'm aware.

[surajssd]

Why do we have a --kubeconfig-file flag for this? Also the help says that it has a fallback mechanism, so it is confusing to the user. I understand the technicalities behind why it shows up here, but then for a user it can be confusing. Either we fail if we have a flag mentioned or this flag should not be here:

$ lokoctl health -h
Get the health of a cluster

Usage:
  lokoctl health [flags]

Flags:
      --debug   Print debug messages
  -h, --help    help for health

Global Flags:
      --kubeconfig-file string   Path to a kubeconfig file. If empty, the following precedence order is used:
                                   1. Cluster asset dir when a lokocfg file is present in the current directory.
                                   2. KUBECONFIG environment variable.
                                   3. ~/.kube/config file.
...

[invidian]

Why do we have a --kubeconfig-file flag for this? Also the help says that it has a fallback mechanism, so it is confusing to the user. I understand the technicalities behind why it shows up here, but then for a user it can be confusing. Either we fail if we have a flag mentioned or this flag should not be here:

Because it's a global flag... :( And yes, entire CLI code should probably be reworked to clean those things up..

[invidian]

@surajssd I pushed the fix with moving --kubeconfig-file flag. Please have a look.

[surajssd]

Verified the lokoctl health functionality works.