Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

upgrading to v0.31.3 results in a 404 #2884

Closed
1 task done
masterkain opened this issue Jun 29, 2022 · 6 comments
Closed
1 task done

upgrading to v0.31.3 results in a 404 #2884

masterkain opened this issue Jun 29, 2022 · 6 comments

Comments

@masterkain
Copy link

masterkain commented Jun 29, 2022 •
edited
Loading

Describe the bug

today on one of my clusters (EKS) I've seen this during an upgrade from v0.31.1:

✔ cloned repository
► generating component manifests
✔ generated component manifests
✔ component manifests are up to date
✔ reconciled components
► determining if source secret "flux-system/flux-system" exists
✔ source secret up to date
► generating sync manifests
✔ generated sync manifests
✔ sync manifests are up to date
► applying sync manifests
✔ reconciled sync configuration
◎ waiting for Kustomization "flux-system/flux-system" to be reconciled
✗ failed to download artifact from http://source-controller.flux-system.svc.cluster.local./gitrepository/flux-system/flux-system/e36b5ede775cff77408cdc3b51091ddc3d666dd0.tar.gz, status: 404 Not Found
► confirming components are healthy
✔ helm-controller: deployment ready
✔ image-automation-controller: deployment ready
✔ image-reflector-controller: deployment ready
✔ kustomize-controller: deployment ready
✔ notification-controller: deployment ready
✔ source-controller: deployment ready
✔ all components are healthy
✗ bootstrap failed with 1 health check failure(s)

not sure if it does any harm but maybe it's worth reporting, it keeps doing that on further attempts

Steps to reproduce

after upgrading flux locally:

flux bootstrap github --context=mycontext --owner=myowner --repository=cluster --path=clusters/mycluster --branch=main --read-write-key --components-extra=image-reflector-controller,image-automation-controller

Expected behavior

no failed health checks

Screenshots and recordings

No response

OS / Distro

macOS 12.4

Flux version

v0.31.3

Flux check

► checking prerequisites
✔ Kubernetes 1.22.9-eks-a64ea69 >=1.20.6-0
► checking controllers
✔ helm-controller: deployment ready
► ghcr.io/fluxcd/helm-controller:v0.22.1
✔ image-automation-controller: deployment ready
► ghcr.io/fluxcd/image-automation-controller:v0.23.4
✔ image-reflector-controller: deployment ready
► ghcr.io/fluxcd/image-reflector-controller:v0.19.2
✔ kustomize-controller: deployment ready
► ghcr.io/fluxcd/kustomize-controller:v0.26.2
✔ notification-controller: deployment ready
► ghcr.io/fluxcd/notification-controller:v0.24.0
✔ source-controller: deployment ready
► ghcr.io/fluxcd/source-controller:v0.25.9
► checking crds
✔ alerts.notification.toolkit.fluxcd.io/v1beta1
✔ buckets.source.toolkit.fluxcd.io/v1beta2
✔ gitrepositories.source.toolkit.fluxcd.io/v1beta2
✔ helmcharts.source.toolkit.fluxcd.io/v1beta2
✔ helmreleases.helm.toolkit.fluxcd.io/v2beta1
✔ helmrepositories.source.toolkit.fluxcd.io/v1beta2
✔ imagepolicies.image.toolkit.fluxcd.io/v1beta1
✔ imagerepositories.image.toolkit.fluxcd.io/v1beta1
✔ imageupdateautomations.image.toolkit.fluxcd.io/v1beta1
✔ kustomizations.kustomize.toolkit.fluxcd.io/v1beta2
✔ providers.notification.toolkit.fluxcd.io/v1beta1
✔ receivers.notification.toolkit.fluxcd.io/v1beta1
✔ all checks passed

Git provider

GitHub

Container Registry provider

No response

Additional context

No response

Code of Conduct

  • I agree to follow this project's Code of Conduct
@timothystewart6
Copy link

I have the same issue on an upgrade

kustomization/core.flux-system
failed to download artifact from http://source-controller.flux-system.svc.cluster.local./gitrepository/flux-system/flux-system/985578b4b6083839811d59477e92d40d83654db2.tar.gz, status: 404 Not Found
revision
master/985578b4b6083839811d59477e92d40d83654db2

@stefanprodan
Copy link
Member

Can you please post here flux get source git flux-system and flux get ks flux-system

@darkowlzz
Copy link
Contributor

Hi, if this is still happening, can you also share the status of the GitRepository object with kubectl -n flux-system get gitrepo flux-system -o yaml. The status can provide more information about the state of the source artifact, if there's any issue in that.

@stefanprodan stefanprodan mentioned this issue Jun 30, 2022
Merged
@masterkain
Copy link
Author

masterkain commented Jun 30, 2022 •
edited
Loading

I tried again today and got no errors, anyway:

kain@kain-osx cluster % flux get source git flux-system
NAME       	REVISION    	SUSPENDED	READY	MESSAGE
flux-system	main/baff7f0	False    	True 	stored artifact for revision 'main/baff7f03dcfdd6635be4de0e9aedb272aee8209f'

kain@kain-osx cluster % flux get ks flux-system
NAME       	REVISION    	SUSPENDED	READY	MESSAGE
flux-system	main/baff7f0	False    	True 	Applied revision: main/baff7f0

kain@kain-osx cluster % kubectl -n flux-system get gitrepo flux-system -o yaml
apiVersion: source.toolkit.fluxcd.io/v1beta2
kind: GitRepository
metadata:
  annotations:
    reconcile.fluxcd.io/requestedAt: 2022-06-30 21:21:12.017358735 +0000 UTC m=+1287229.872381964
  creationTimestamp: "2022-05-14T23:17:22Z"
  finalizers:
  - finalizers.fluxcd.io
  generation: 1
  labels:
    kustomize.toolkit.fluxcd.io/name: flux-system
    kustomize.toolkit.fluxcd.io/namespace: flux-system
  name: flux-system
  namespace: flux-system
  resourceVersion: "37038373"
  uid: 2e7d7c92-a9a0-42ef-8104-ea5d9c3bb9c7
spec:
  gitImplementation: go-git
  interval: 1m0s
  ref:
    branch: main
  secretRef:
    name: flux-system
  timeout: 60s
  url: ssh://git@github.com/xxx/xxx
status:
  artifact:
    checksum: 5035720dafc4325397bb733eb411673fc0a470ba4f6b88da468769ce0adda748
    lastUpdateTime: "2022-06-30T21:21:14Z"
    path: gitrepository/flux-system/flux-system/baff7f03dcfdd6635be4de0e9aedb272aee8209f.tar.gz
    revision: main/baff7f03dcfdd6635be4de0e9aedb272aee8209f
    size: 944967
    url: http://source-controller.flux-system.svc.cluster.local./gitrepository/flux-system/flux-system/baff7f03dcfdd6635be4de0e9aedb272aee8209f.tar.gz
  conditions:
  - lastTransitionTime: "2022-06-30T21:21:14Z"
    message: stored artifact for revision 'main/baff7f03dcfdd6635be4de0e9aedb272aee8209f'
    observedGeneration: 1
    reason: Succeeded
    status: "True"
    type: Ready
  - lastTransitionTime: "2022-06-30T21:21:14Z"
    message: stored artifact for revision 'main/baff7f03dcfdd6635be4de0e9aedb272aee8209f'
    observedGeneration: 1
    reason: Succeeded
    status: "True"
    type: ArtifactInStorage
  contentConfigChecksum: sha256:fcbcf165908dd18a9e49f7ff27810176db8e9f63b4352213741664245224f8aa
  lastHandledReconcileAt: 2022-06-30 21:21:12.017358735 +0000 UTC m=+1287229.872381964
  observedGeneration: 1
  url: http://source-controller.flux-system.svc.cluster.local./gitrepository/flux-system/flux-system/latest.tar.gz

@timothystewart6
Copy link

timothystewart6 commented Jun 30, 2022 •
edited
Loading

Here's my output

➜  home-ops git:(master) flux get source git flux-system
NAME            REVISION        SUSPENDED       READY   MESSAGE

flux-system     master/0fc8d21  False           True    stored artifact for revision 'master/0fc8d21831f141228440ab1949edf60ba3120001'

➜  home-ops git:(master) flux get ks flux-system
NAME            REVISION        SUSPENDED       READY   MESSAGE

flux-system     master/0fc8d21  False           True    Applied revision: master/0fc8d21

The error stopped around 6/30/2022 10:30 CST

@stefanprodan
Copy link
Member

This was fixed in Flux 0.32.0

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@masterkain @darkowlzz @timothystewart6 @stefanprodan