kubernetes version changed when stop/start Colima #417

prune998 · 2022-09-06T23:59:43Z

Description

Reproducing:

start colima with a specific k8s version using colima start --edit and set version to v1.25.0-rc1+k3s1 or colima start --kubernetes-version v1.25.0-rc1+k3s1
Validate the version in the config file:

grep "  version"  ~/.colima/default/colima.yaml
  version: v1.25.0-rc1+k3s1

k version
...
Server Version: version.Info{Major:"1", Minor:"25", GitVersion:"v1.25.0-rc1+k3s1", ...

stop colima with colima stop
restart colima without any change to the config with colima start
version is changed to default v1.23.6+k3s1
you can see this in the logs:

❯ colima start
INFO[0000] starting colima
INFO[0000] runtime: containerd+k3s
INFO[0000] preparing network ...                         context=vm
INFO[0000] starting ...                                  context=vm
INFO[0030] provisioning ...                              context=containerd
INFO[0030] starting ...                                  context=containerd
INFO[0036] provisioning ...                              context=kubernetes
INFO[0036] version changed to v1.23.6+k3s1, downloading and installing ...  context=kubernetes  <-----------
INFO[0048] loading oci images ...                        context=kubernetes
INFO[0056] starting ...                                  context=kubernetes
INFO[0068] done

❯ grep "  version"  ~/.colima/default/colima.yaml
  version: v1.23.6+k3s1

Version

Colima Version:

❯ colima version
colima version 0.4.4
git commit: 8bb1101a861a8b6d2ef6e16aca97a835f65c4f8f

runtime: containerd
arch: x86_64
client: v0.20.0
server: v1.5.8

kubernetes
Client Version: v1.25.0
Kustomize Version: v4.5.7
Server Version: v1.23.6+k3s1

Lima Version:

❯ limactl -v
limactl version 0.11.3

Qemu Version: na

Operating System

macOS Intel
macOS M1
Linux

Reproduction Steps

colima start --kubernetes-version v1.25.0-rc1+k3s1
INFO[0000] starting colima
INFO[0000] runtime: containerd+k3s
INFO[0000] preparing network ...                         context=vm
INFO[0000] starting ...                                  context=vm
INFO[0030] provisioning ...                              context=containerd
INFO[0030] starting ...                                  context=containerd
INFO[0036] provisioning ...                              context=kubernetes
INFO[0036] downloading and installing ...                context=kubernetes
INFO[0047] loading oci images ...                        context=kubernetes
INFO[0055] starting ...                                  context=kubernetes
INFO[0061] updating config ...                           context=kubernetes
INFO[0062] Switched to context "colima".                 context=kubernetes
INFO[0062] done


grep "  version"  ~/.colima/default/colima.yaml
  version: v1.25.0-rc1+k3s1

colima stop
INFO[0000] stopping colima
INFO[0000] stopping ...                                  context=kubernetes
INFO[0001] stopping ...                                  context=containerd
INFO[0003] stopping ...                                  context=vm
INFO[0007] done


colima start
INFO[0000] starting colima
INFO[0000] runtime: containerd+k3s
INFO[0000] preparing network ...                         context=vm
INFO[0000] starting ...                                  context=vm
INFO[0030] provisioning ...                              context=containerd
INFO[0030] starting ...                                  context=containerd
INFO[0036] provisioning ...                              context=kubernetes
INFO[0036] version changed to v1.23.6+k3s1, downloading and installing ...  context=kubernetes
INFO[0048] loading oci images ...                        context=kubernetes
INFO[0057] starting ...                                  context=kubernetes
INFO[0063] done


grep "  version"  ~/.colima/default/colima.yaml
  version: v1.23.6+k3s1

Expected behaviour

I would expect the K8s version to stay the same after a stop/start
I would expect that Colima will not change the default profile after its creation and editing

Additional context

No response

The text was updated successfully, but these errors were encountered:

abiosoft · 2022-09-18T07:41:35Z

Thanks for reporting. I would try to reproduce on my end.

killwing · 2022-10-12T11:15:14Z

I also can not change the version by kubernetes.version of colima.yaml.
After I updating colima.yaml and colima k delete & colima k start, the version is still v1.25.0+k3s1.

colima version
colima version 0.4.6
git commit: 10377f3

runtime: containerd
arch: x86_64
client: v0.23.0
server: v1.6.8

kubernetes
Client Version: v1.23.2
Server Version: v1.25.0+k3s1

abiosoft · 2022-10-13T04:31:05Z

@killwing thanks for confirming. This is being prioritised.

abiosoft · 2022-10-13T06:57:11Z

@prune998 @killwing can you kindly confirm if this is now fixed? You can test with the development version brew install --head colima.

prune998 · 2022-10-13T12:12:27Z

will do ASAP. Thanks @abiosoft

prune998 · 2022-10-13T12:41:49Z

I do see an error, not related to this issue, but reporting it to keep track:

❯ colima start --runtime containerd --with-kubernetes --kubernetes-version v1.24.4+k3s1
INFO[0000] starting colima
INFO[0000] runtime: containerd+k3s
INFO[0000] preparing network ...                         context=vm
INFO[0000] starting ...                                  context=vm
> Using the existing instance "colima"
> "Non-strict YAML is deprecated and will be unsupported in a future version of Lima" comment="main file \"/Users/prune/.lima/colima-k8s-1-25/lima.yaml\"" error="[93:1] unknown field \"colimaState\"\n

Beside that I can confirm the K8s version stays the same:

❯ colima stop
INFO[0000] stopping colima
INFO[0000] stopping ...                                  context=kubernetes
INFO[0002] stopping ...                                  context=containerd
INFO[0003] stopping ...                                  context=vm
INFO[0009] done

❯ grep "  version"  ~/.colima/default/colima.yaml
  version: v1.24.4+k3s1

❯ colima start
INFO[0000] starting colima
INFO[0000] runtime: containerd+k3s
INFO[0000] preparing network ...                         context=vm
INFO[0000] starting ...                                  context=vm
INFO[0028] provisioning ...                              context=containerd
INFO[0028] starting ...                                  context=containerd
INFO[0033] provisioning ...                              context=kubernetes
INFO[0035] starting ...                                  context=kubernetes
INFO[0039] done

❯ grep "  version"  ~/.colima/default/colima.yaml
  version: v1.24.4+k3s1

If I then update the config file and change the version:

❯ vi .colima/default/colima.yaml
# change version to  v1.25.0-rc1+k3s1,

❯ colima start
INFO[0000] starting colima
INFO[0000] runtime: containerd+k3s
INFO[0000] preparing network ...                         context=vm
INFO[0000] starting ...                                  context=vm
INFO[0030] provisioning ...                              context=containerd
INFO[0030] starting ...                                  context=containerd
INFO[0036] provisioning ...                              context=kubernetes
INFO[0036] version changed to v1.25.0-rc1+k3s1, downloading and installing ...  context=kubernetes
INFO[0047] loading oci images ...                        context=kubernetes
INFO[0057] starting ...                                  context=kubernetes
INFO[0063] done

Everything is working as expected !
Thanks @abiosoft you can close this issue !

abiosoft · 2022-10-13T12:57:29Z

@prune998 thanks for verifying. I might do a quick release as this is a notable bug.

As for the error, you can ignore it. It is due to compatibility with Lima v0.12.0 and it is just a warning.
You should get the error at most once, the Lima config gets (re)generated on each startup.

OliHarris · 2022-10-24T09:35:08Z

Hi there again @abiosoft - was going to say thanks for this amazing software!
I am also seeing this bug now basically on my M1 Mac - running colima start wipes the file -~/.colima/default/colima.yaml each time.

Here is my:
colima version

colima version 0.4.6
git commit: 10377f3a20c2b0f7196ad5944264b69f048a3d40

runtime: docker
arch: x86_64
client: v20.10.20
server: v20.10.18

abiosoft · 2022-10-24T09:50:44Z

@OliHarris yeah, it does read and regenerate the file on startup, but the settings persists. Do you add extra comments to the file?

OliHarris · 2022-10-24T10:01:26Z

Yes indeed, I update values in ~/.colima/default/colima.yaml then run colima start - this file then reverts back to default settings though. Currently I'm looking to install v0.4.5:

brew extract --version=0.4.5  colima homebrew/cask
brew install homebrew/cask/colima@0.4.5

However I get error:
Error: Failure while executing; `\{\"SHELL\"=\>\"bash\"\} /opt/homebrew/Cellar/colima@0.4.5/0.4.5/bin/colima completion bash` exited with 127

So I will continue with using v0.4.6. Will manually update the colima template and make sure values persist - I basically need an x86_64 VM on my M1Mac to run the Oracle Database :)

OliHarris · 2022-10-24T10:56:01Z

OK I updated colima template eg arch: x86_64 - and can see these values persist when I run that command only; however running colima start then colima status

INFO[0000] colima is running                            
INFO[0000] arch: aarch64                                
INFO[0000] runtime: docker                              
INFO[0000] mountType: sshfs                             
INFO[0000] socket: unix:///Users/ogharris/.colima/default/docker.sock

So now I will try with the command line eg:
colima start -c 4 -m 12 -a x86_64

OliHarris · 2022-10-24T11:01:40Z

OK can confirm am once again seeing this issue now:
#411

Here is that serial.log again:

�[2J�[01;01H�[=3h�[2J�[01;01HBdsDxe: failed to load Boot0001 "UEFI QEMU DVD-ROM QM00001 " from PciRoot(0x0)/Pci(0x1F,0x2)/Sata(0x0,0xFFFF,0x0): Not Found
BdsDxe: failed to load Boot0002 "UEFI QEMU DVD-ROM QM00005 " from PciRoot(0x0)/Pci(0x1F,0x2)/Sata(0x2,0xFFFF,0x0): Not Found
BdsDxe: failed to load Boot0003 "UEFI Misc Device" from PciRoot(0x0)/Pci(0x8,0x0): Not Found

>>Start PXE over IPv4.
  PXE-E16: No valid offer received.
BdsDxe: failed to load Boot0004 "UEFI PXEv4 (MAC:5255556DF58D)" from PciRoot(0x0)/Pci(0x1,0x0)/MAC(5255556DF58D,0x1)/IPv4(0.0.0.0,0x0,DHCP,0.0.0.0,0.0.0.0,0.0.0.0): Not Found

>>Start PXE over IPv6.

  PXE-E16: No valid offer received.
BdsDxe: failed to load Boot0005 "UEFI PXEv6 (MAC:5255556DF58D)" from PciRoot(0x0)/Pci(0x1,0x0)/MAC(5255556DF58D,0x1)/IPv6(0000:0000:0000:0000:0000:0000:0000:0000,0x0,Static,0000:0000:0000:0000:0000:0000:0000:0000,0x40,0000:0000:0000:0000:0000:0000:0000:0000): Not Found

>>Start HTTP Boot over IPv4..
.
.
.

  Error: Could not retrieve NBP file size from
 HTTP server.

  Error: Server response timeout.
BdsDxe: failed to load Boot0006 "UEFI HTTPv4 (MAC:5255556DF58D)" from PciRoot(0x0)/Pci(0x1,0x0)/MAC(5255556DF58D,0x1)/IPv4(0.0.0.0,0x0,DHCP,0.0.0.0,0.0.0.0,0.0.0.0)/Uri(): Not Found

>>Start HTTP Boot over IPv6.

  Error: Could not retrieve NBP file size from HTTP server.

  Error: Unexpected network error.
BdsDxe: failed to load Boot0007 "UEFI HTTPv6 (MAC:5255556DF58D)" from PciRoot(0x0)/Pci(0x1,0x0)/MAC(5255556DF58D,0x1)/IPv6(0000:0000:0000:0000:0000:0000:0000:0000,0x0,Static,0000:0000:0000:0000:0000:0000:0000:0000,0x40,0000:0000:0000:0000:0000:0000:0000:0000)/Uri(): Not Found

>>Start PXE over IPv4.

abiosoft · 2022-10-24T11:08:29Z

@OliHarris we should probably have better documentation.

With regards to the config issue,
you cannot modify an architecture of an existing VM. It will always revert to the initial architecture.

OliHarris · 2022-10-24T11:20:05Z

@abiosoft - thanks for the update.
I'm now trying v0.4.5 binary install:

curl -LO https://github.com/abiosoft/colima/releases/download/v0.4.5/colima-$(uname)-$(uname -m)
sudo install colima-$(uname)-$(uname -m) /usr/local/bin/colima

This does seem to honour the colima.yaml file values; however stills to not get passed SSH stage with colima start, when trying to load arch: x86_64

I'll try decrementing releases until I can see this working :)

OliHarris · 2022-10-24T11:55:08Z

@abiosoft as an update - I've tried every version released from v0.4.6 down to v0.3.4 and I'm still seeing SSH error whenever I try to boot arch: x86_64 architecture from my M1 Mac.
Will investigate further...

abiosoft · 2022-10-24T11:59:42Z

I would recommend you delete the VM before trying again (provided you're not doing that).

And also for you to use extra profiles for experiments, which makes it easy to discard.

OliHarris · 2022-10-24T12:01:45Z

Presumably colima delete will do that?
I've gone back to v0.4.5 - as said this does seem to honour the colima.yaml file values; however stills to not get passed SSH stage with colima start, when trying to load arch: x86_64

OliHarris · 2022-10-24T14:17:49Z

Hi @abiosoft - not sure how or why - I'm using v0.4.5 and by doing:

colima delete
colima start --arch amd

Checking colima status for arch: x86_64
colima stop
Then updating adding back the non-default parameters to my colima.yaml file ie arch: x86_64

Things seem to be working!

abiosoft · 2022-10-24T16:27:33Z

Hi @abiosoft - not sure how or why - I'm using v0.4.5 and by doing:
colima delete
colima start --arch amd
Checking colima status for arch: x86_64 colima stop Then updating adding back the non-default parameters to my colima.yaml file ie arch: x86_64

Things seem to be working!

amd, amd64, x86, x64 are all aliases for x86_64 😉

Because cli flags can be specified as well, the config is combined with the flags and re-generated on each startup.
Due to that, some aliases may get rewritten to the actual value in the config.

abiosoft added the bug Something isn't working label Sep 18, 2022

abiosoft mentioned this issue Oct 13, 2022

vm: multiple minor fixes #442

Merged

abiosoft closed this as completed in #442 Oct 13, 2022

abiosoft mentioned this issue Nov 18, 2022

template: save default config from template #485

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

kubernetes version changed when stop/start Colima #417

kubernetes version changed when stop/start Colima #417

prune998 commented Sep 6, 2022

abiosoft commented Sep 18, 2022

killwing commented Oct 12, 2022 •

edited

Loading

abiosoft commented Oct 13, 2022

abiosoft commented Oct 13, 2022

prune998 commented Oct 13, 2022

prune998 commented Oct 13, 2022

abiosoft commented Oct 13, 2022 •

edited

Loading

OliHarris commented Oct 24, 2022

abiosoft commented Oct 24, 2022

OliHarris commented Oct 24, 2022 •

edited

Loading

OliHarris commented Oct 24, 2022 •

edited

Loading

OliHarris commented Oct 24, 2022

abiosoft commented Oct 24, 2022

OliHarris commented Oct 24, 2022

OliHarris commented Oct 24, 2022

abiosoft commented Oct 24, 2022

OliHarris commented Oct 24, 2022

OliHarris commented Oct 24, 2022

abiosoft commented Oct 24, 2022

kubernetes version changed when stop/start Colima #417

kubernetes version changed when stop/start Colima #417

Comments

prune998 commented Sep 6, 2022

Description

Version

Operating System

Reproduction Steps

Expected behaviour

Additional context

abiosoft commented Sep 18, 2022

killwing commented Oct 12, 2022 • edited Loading

abiosoft commented Oct 13, 2022

abiosoft commented Oct 13, 2022

prune998 commented Oct 13, 2022

prune998 commented Oct 13, 2022

abiosoft commented Oct 13, 2022 • edited Loading

OliHarris commented Oct 24, 2022

abiosoft commented Oct 24, 2022

OliHarris commented Oct 24, 2022 • edited Loading

OliHarris commented Oct 24, 2022 • edited Loading

OliHarris commented Oct 24, 2022

abiosoft commented Oct 24, 2022

OliHarris commented Oct 24, 2022

OliHarris commented Oct 24, 2022

abiosoft commented Oct 24, 2022

OliHarris commented Oct 24, 2022

OliHarris commented Oct 24, 2022

abiosoft commented Oct 24, 2022

killwing commented Oct 12, 2022 •

edited

Loading

abiosoft commented Oct 13, 2022 •

edited

Loading

OliHarris commented Oct 24, 2022 •

edited

Loading

OliHarris commented Oct 24, 2022 •

edited

Loading