Meeting

Flux dev meeting

The Flux developer community holds public meetings to discuss where Flux is going, to get feedback and to create an opportunity to get to know each other face-to-face and in a more immediate fashion than on Slack.

The next meeting is going to take place:

• When: 2019-03-26 17:00 UTC
• Where: Zoom

Before joining, make sure you install the Zoom client and test Microphone/video settings. If you prefer not to be on camera, or just want to listen in, that's absolutely fine too.

Agenda and notes

Next meeting: 2019-03-26 17:00 UTC

Agenda

• Review of actions from last meeting
  • Fons to send proposal to mailing list too (configuration generators).
  • Folks to reply on the mailing list to (Justin's) proposal.
  • Bring up Helm Op 1.0 on mailing list, ask for feedback. (Opportunity to ask people for testing before we release.)
  • Fons (and Stefan?) to talk to James to figure out the onboarding of Windows package submission. (Michael reminded everyone of more and more Azure people coming in, so quite some interest in Windows.) Folks to coordinate on Slack (and/or the mailing list).
  • Michael/Stefan/Justin to discuss integration tests on the mailing list or Slack. Stefan to start discussion about using Kind (discussion will be: which tests to use, most scripts for testing should be already there). Nick has written skaffolding for running tests already.
  • ✅ Daniel to plan next meeting.
• Helm-op 1.0 status
• Next meeting (DST change, etc?)

Minutes

• Attendees:
• Video upload:
• Minutes: <add notes here>

Past meetings

• 2019-02-26
• 2018-11-28

2019-02-26

Agenda

• Review of actions from last meeting
• Flux roadmap
• Integration tests, next steps
• Plan next meeting

Minutes

• Attendees: Alfonso Acosta, Daniel Holbach, Hidde Beydals, James Pruitt, Justin Barrick, Michael Bridgen, Stefan Prodan, Tamao Nakahara
• Video upload: https://zoom.us/recording/share/2GOFW8UGk7Lu9ob5luUkp6-z8FgVU3BkksTv_QjI8_qwIumekTziMw
• Minutes:
  • Introductions:
    • Justin Barrick, works at Mesosphere, has been contributing "off and on" to Flux and is maintainer of Flux.
    • Alfonso (Fons) Acosta, recently rejoined Weaveworks, works on Flux, not familiar with all the details yet. Very excited about Weave Flux and its activity and community. The vibe is great too.
  • Review of action items
    • ✅ Daniel/Michael to put together strawman for helm-op-1.0 release. TBD in separate "Flux Roadmap" agenda item.
    • Integrations tests, TBD in separate agenda item.
    • ✅ Set up meeting.
    • ✅ Set up mailing list. Here came a reminder for everyone to subscribe to the flux-dev mailing list. It's not meant to replace Slack, but to allow for longer, more structured conversations.
  • Flux Roadmap
    • Michael made clear that there's no published roadmap, but that there is a list of things which are being worked on that are realistic to land soon.
    • Big ticket items to land soon:
      • Signing and fully verifying commits. First half has landed (Hidde): signing everything (sync tag and commits get signed) is done, verification needs to be done still, is about 25% of the remaining part. Alfonso asked if it's a backwards compatibility problem, Hidde clarified that it's behind a flag which needs to be enabled and will only verify commits which land after enabling the flag. Daniel asked if it was going to be turned on by default at some stage. This will be decided after evaluating the user experience because it requires manual steps.
      • Garbage collection of resources. In a good state, to be landed soon. It brings us much to closer to "Flux reflects the state in git in the cluster".
      • Allowing certain namespaces. We'll be graduating this feature from experimental to GA by extending it to cover everything. (Fons worked on this. A couple more weeks.)
    • Experimental, or not started yet:
      • Readonly mode (sync only, don't write sync-tag).
      • Integrate configuration generators into Flux. Not just flat YAML files, e.g. ksonnet/kustomize users. Fons has been working on a PoC, or rather writing a design spec. This is getting even more relevant as kustomize is likely going to get merged into kubernetes.
        • ACTION: Fons to send proposal to mailing list too.
    • Comments and expected changes/features to land in the next months
      • Justin sent a post to the dev list about writing changes in the cluster back to git. His idea was having two daemons, one which would write changes back to git. It would have to live in Flux git, but Justin wasn't sure how people would feel about it. It would make things easier, more composable and enable interesting CI/CD interactions. It'd be quite a bit of work. Fons asked for general overview and a few use-cases. Justin explained that Flux would have rules and heuristics, which would allow Flux to figure out from Kubernetes which changes to commit to git. Flux does this for images already. This would be more generic. Justin's use-case was a backup workflow using the VolumeSnapshot API. He wanted Flux to write the resources to Git, so if the cluster blows up, things could be easily restored (he has an operator which does the backup already, but the information needed to be made persistent). Fons asked if this was going to apply to every resource in the cluster. Justin said it would depend on the heuristics (e.g. applied by kubectl?, change to an annotation, or through a deployment?). Michael added: some bits of manifests like checkpointing HPA data back to git, so it doesn't always start by 0. It'd be hard though to generally rule out feedback loops, so causality only flows one way. Fons wondered if kubediff could be tied in and there'd be a way to easily reconcile changes. The idea being to provide people with better tools to resolve these differences, e.g. "no, it wasn't accident" vs. "yes, please persist this change". A "fluxctl diff" could show you the differences. Stefan suggested that a "dry-run" option could test this, using the new dry-run API (beta in Kubernetes 1.13).
        • ACTION: Folks to reply on the mailing list to proposal.
      • Helm Operator 1.0 roadmap: shopping list of items on the list, we want comments.
        • Bump API version to v1 (Kubernetes CRD version).
        • Put a hook into Helm Operator, so you can poke it and tell it to look at Git (this is in fluxd already).
        • Extend supported formats of images in Helm charts.
        • Break out Helm operator into its own repository. (We'll at least attempt this.)
        • Feedback and questions:
          • What to do if Helm moves to v3? 1) No Tiller, everything client-side. 2) Move to CRD model. Stefan explained which bits will change (don't talk to Tiller any more, let Helm client bootstrap, query CRD through Kubernetes). It will be an interesting challenge to support v2 and v3 and ensure compatibility.
          • Stefan explained that an interesting change will land in Helm (probably backported to Helm v2): use Docker registries for Helm repositories. Maybe use in Helm Operator 1.1?
        • ACTION: to bring up Helm Op 1.0 on mailing list, ask for feedback. (Opportunity to ask people for testing before we release.)
  • fluxctl in Chocolatey: James worked on getting the Windows binary into Chocolatey (Github issue). Binary build and verification process should be owned the Weaveworks org, right now it's in James's Github (link). It'd be great to open this up to more folks and automate it. Testing packages is a bit hard because of provisioning Windows VM where you can install, uninstall and add more unit tests, e.g. virus-scanning or whatever else. KVM can be used, Hyper-V too. There would be the option to leave the verification step to the Chocolatey community. Stefan asked about Appveyor (CI for .NET). Chocolatey seems to use Vagrant, which does not work on Appveyor. Fons has Appveyor experience too - they seem to provide communities with one VM for tests and have pre-built VMs.
    • ACTION: Fons (and Stefan?) to talk to James to figure out the onboarding of Windows package submission. (Michael reminded everyone of more and more Azure people coming in, so quite some interest in Windows.) Folks to coordinate on Slack (and/or the mailing list).
    • Some discussion about Kubernetes entirely on Windows. Stefan clarified that Azure would deploy Flux on a Linux node.
    • Some discussion about Arm builds, some folks pointed towards drone.io. The question: do we want to add build profiles for Arm, Windows, etc.?
  • Integration tests, next steps: the PR has been sitting around for a bit. We ran into issues because it required Kubernetes in an already containerised environment. There was discussion about Docker-on-Docker, or running minikube in CircleCI. Stefan built end-to-end testing of flagger in CircleCI using Kubernetes Kind, which uses Istio and tons of things. Using just Flux should be loads easier - just use a small git server. Justin uses end-to-end test of flux-operator (and other things) in Drone using localkube, but would go the Kubernetes Kind route today. Stefan recommended CircleCI because we already use it and it has enough memory.
    • ACTION: Michael/Stefan/Justin to discuss this on the mailing list or Slack. Stefan to start discussion about using Kind (discussion will be: which tests to use, most scripts for testing should be already there). Nick has written skaffolding for running tests already.
  • Next meeting: agreement on once per month.
    • ACTION: Daniel to plan next meeting.

2018-11-28

Agenda

This is going to be the first meeting, we'll want to get to know each other in our community and get feedback. If you're interested in joining the community, this will be a good place to get involved.

1. Quick introductions (who are you, how do you use Flux)
   If you prefer not to introduce yourself, or just listen in, that's totally fine!
2. State of Flux (Michael)
3. Future plans:
   1. Helm Operator 1.0
   2. Themed releases
   3. Integration test machinery
4. Feedback and questions
5. Next meeting

Minutes

• Attendees: Brant Bobby, Daniel Holbach, Hidde Beydals, Michael Bridgen, Nick Cabatoff, Stefan Prodan
• Video upload: https://zoom.us/recording/play/j3XnvYfXYunpobpanrDZ3F-UarqTye7L0QNk9zS04HjNQR8Aqj5OCSg2D47VzQhq?continueMode=true
• Minutes:
  • Introductions:
    • Michael Bridgen, London, Weaveworks, Lead Dev on Flux, RabbitMQ/VMWare before. Mostly works on Flux.
    • Daniel Holbach, Berlin, Weaveworks, Community Manager, Ubuntu dev communities before. Wants to bring more fun, and a bit more structure into Flux dev community.
    • Nick Cabatoff, used to work at Intelerad, was looking for a CD system, discovered Flux, liked it, decided to get more involved. Moved on to Hashicorp recently.
    • Brant Bobby, works in Canada, looks into transitioning to Kubernetes as a greenfield project. Flux looks like the best option for a CD system there. Is a very enthusiastic user.
    • Hidde Beydals, his story is very similar to Brant's. (Hidde's connection dropped here.)
    • Stefan Prodan, Weaveworks, Developer Experience team. Works on Flux and GitOps.
  • State of Flux (Michael):
    • Flux is not just a Weaveworks project any more. Hidde Beydals, Justin Barrick, Nick Cabatoff joined as maintainers.
    • Flux is used as part of Weaveworks's commercial Weave Cloud product. Rollout progress was added to WC's UI. A PR is in progress to add this to fluxctl release.
    • Another PR that's in the works: Garbage collection for objects which are not part of the Git repo any more.
    • Helm operator: still in beta, but steadily improving. Sees a lot of love through PRs and issues. Let's try to get it to its "1.0" release. It's what multiplied attention to Flux.
    • Supporting syncing multiple Git repos. Very popular request.
    • Support thing like kustomize natively or support composition of CI tooling. Ideas and help welcome!
  • Plans/request/ideas:
    • Helm operator 1.0 - what's missing?
      • Secrets for getting keys for each helm release
      • Some issues open for polish and reliability.
      • Making it less chatty in the logs.
      • Make it follow all of the design spec.
      • No huge drawbacks at this point. Definitely covers the 80%.
      • Let's review open issues and put together a shopping list. Review together. Make it theme of the next set of releases. Ask people to test. Get feedback.
      • ACTION: Daniel/Michael to put together straw-man.
      • Discussion:
        • Hidde: works fine in their setting.
        • Brant: works Helm for external apps, not own apps.
        • Hidde: Use Helm for launching apps with different settings.
        • Brant: Still figuring out how to use Helm for "multi-user support".
        • Somewhat unclear how people organise their environments. Flux needs to stay flexible.
        • Stefan: Flux per namespaces/teams. Clearer structure which responsibilities are with Flux and which are for the helm-operator.
        • Stefan: simple CICD pipeline with Helm releases and Github Actions are possible now.
        • Michael: It'd be great to make Flux more usable with CI pipelines. Waiting for stuff to happen is not easy with Kubernetes.
        • Stefan: use Progress Deadlines? Need a timeout somewhere. Depending on what kind of workload (StatefulSet or not).
        • Michael: let Flux honour deadline, otherwise use Flux setting. Needs design.
        • Stefan: this enables to give feedback to e.g. Github.
        • Michael: previous work on notifications on rollout status (Roli, Elena).
    • Themed releases, ideas
      • User experience: small things like making fluxctl installs easier, crystal-clear log messages.
      • Docs/tutorials: e.g. automated annotations.
    • Integration tests
      • Nick worked on this, some discussion. Let's revive this work. Would be great to get contributors involved with writing tests too. Michael helped with this as well. Raise confidence in not causing regressions.
      • ACTION: Michael/Nick to see what's still missing.
      • Stefan: let's use flux-get-started(?), it covers e.g. all kinds of annotations.
      • Stefan: Damien wrote code to make integration test automations easy. It's used in eksctl.
    • Next meeting
      • ACTION: Daniel: find out on Slack if we need meeting time? Target early next year.
      • Ask for agenda items ahead of time. Try monthly for now. (We compared to Scope's fortnightly meetings.)
      • Focus on design decisions and feedback as opposed to reviewing issues/PRs there.
    • Mailing list
      • Let's use one for long-form design decisions. Slack not really good for asynchronous discussions, not good for planning - some people prefer not to use Slack.
      • ACTION: Michael to set up mailing list.
    • Thanks
      • Thanks to the new maintainers who stepped up!
      • Thanks to everyone who helped out in the past weeks. Flux has got a really great team together!