Discussing/modifying the kubernetes/kubernetes release cadence #1290
Replies: 85 comments 4 replies
-
|
I am a +1 for 3 releases/year. As you noted, this will allow us to work on items that can enhance things |
Beta Was this translation helpful? Give feedback.
-
|
My personal opinion is that three releases per year are enough. This means more time can be spent on actual feature development, without narrowing them down just to fit into the release cycle. It will also reduce the management overhead for SIG Release (and the release engineering). On the other hand it will give less people a chance to participate in the shadowing program for example. Anyways, I think we will find appropriate solutions around those kind of new challenges. |
Beta Was this translation helpful? Give feedback.
-
|
I don’t have a strong preference on the one or the other. But from what I
see from our clients or even from public cloud providers, 4 release seems
to be a huge struggle.
Therefore
+1 4 3 (you get joke in it)
|
Beta Was this translation helpful? Give feedback.
-
|
Huge +1. In a series of user surveys from SIG Cluster lifecycle, we've consistently found that users struggle to keep up with 4 releases a year and have data to show this. cc @neolit123 |
Beta Was this translation helpful? Give feedback.
-
|
+1
i find these results surprising. we had the same question in the latest SIG CL survey and the most picked answer was "not sure". this tells me that users possibly do not understand all the implications or it does not matter to them. a couple of benefits that i see from the developer side:
as long as SIG Arch are on board we should just proceed with the change. EDIT: to Tim's point
this however we did see, users struggle to upgrade. |
Beta Was this translation helpful? Give feedback.
-
puts on external consumer hat This would be super great. We really, really struggle as is. takes off external consumer hat
puts on release lead hat This I am 100% in agreement on. When we kicked off 1.18 (because of the extra time over the new year and stuff), we had extra time to solicit shadows and as the enhancements lead for 1.18, I was really able to onboard my shadows early and hit the ground running. For 1.20, we didn't have a large amount of time between identifying an EA, 1.19 ending, and 1.20 starting so selecting the shadows and getting things rolling was a challenge, especially for the front loaded work Enhancements has to do. The one downside is that we will remove an opportunity for shadowing, and as we saw this time around we had >100 people apply, and this will remove ~24-ish opportunities. I think we can maybe identify some opportunities for folks that want to be involved though. takes off release lead hat I think in general, this gives people more time do planning within their SIG, work on KEPS that fall out of that planning, and maybe work toward general health and well being of tests? Probably a lot to work out if we make that decision, but overall I am pretty strongly in favor of this. |
Beta Was this translation helpful? Give feedback.
-
|
With everything going on three releases is fine, we should be considerate of the challenges everyone has right now. |
Beta Was this translation helpful? Give feedback.
-
|
I look forward to using the mechanisms already in place (notably CustomResourceDefinition, but also things like scheduling plugins) to enhance the Kubernetes experience outside of the minor release cycle. A bit more decoupling, now that the investment is made to enable that, sounds good to me - and allows for minor releases of Kubernetes itself to become less frequent. |
Beta Was this translation helpful? Give feedback.
-
|
Big +1 on the 3 releases/year cadence. As the project matures, having less churn becomes a very desirable feature 😉 |
Beta Was this translation helpful? Give feedback.
-
|
++ for 3 releases a year |
Beta Was this translation helpful? Give feedback.
-
|
👍 for 3. |
Beta Was this translation helpful? Give feedback.
-
|
+1 for 3 releases |
Beta Was this translation helpful? Give feedback.
-
|
👍 for 3 releases a year. |
Beta Was this translation helpful? Give feedback.
-
|
Huge +1 for three releases a year. From an end user perspective it puts less toil on the teams to keep up with the upgrades and administrative work associated with it. |
Beta Was this translation helpful? Give feedback.
-
|
(Thanks everyone for the feedback thus far. Keep it coming! 💕) |
Beta Was this translation helpful? Give feedback.
-
|
+1 for three releases a year, and all of this discussion is fantastic! I agree with there being three releases a year. However, I do think that having more regular minor version releases would be helpful, so there isn’t any rush to get things into a specific release, nor a blocker around shipping bugfixes or improvements. I’d like to propose a strongly scoped path for Alpha, Beta, and GA features. I believe that allowing for a bit more leniency for Alpha and Beta code promotion and more stringent requirements for features before they make GA status. |
Beta Was this translation helpful? Give feedback.
-
|
I agree with @spiffxp that whatever we end up doing, we should acknowledge that calendar Q4 is substantially quieter than other quarters, with US Kubecon rolling into US Thanksgiving, rolling into the December festive season. I think that any plan to change the release cadence needs to take that as a prime consideration, whether it's keeping four releases a year and marking the Q4 one as minimal features, spreading three releases across the year, or some other solution. |
Beta Was this translation helpful? Give feedback.
-
|
@spiffxp we've talked about making Q4 a "maintenance" release endlessly, but we've never actually implemented that. |
Beta Was this translation helpful? Give feedback.
-
|
Sounds like joshs comment is middle ground on the way to three : sure you get 4 releases but the fourth is only bug fixes, tests and stability . |
Beta Was this translation helpful? Give feedback.
-
|
I'm in favor of three releases a year. I like @jberkus comment about a Q4 maintenance release too without so much hoopla and fanfare. I hope folks aren't driven to only fix stuff in Q4, "Oh that's a gnarly one, wait until the end of the year." Is something I could foresee someone thinking at some point, if we don't word things right about a maintenance release. One question I think has been lightly touched on is, "What about LTS releases?" (and I know this is out of scope but, I don't know where we stand on this atm) |
Beta Was this translation helpful? Give feedback.
-
|
The consensus on LTS (meaning multi-year support for a single version) is, in short, there's no consensus. We in the LTS WG worked for over two years, and we were able to get everyone to agree to extend the support window to one year (from nine months), which I think speaks to the passion that everyone has about this, the diversity of the use cases Kubernetes is supporting, and the community's determination to get it right. Speaking personally, I think that LTS is a long way away, if ever - it would require a lot more stability in the core than we have right now. With efforts like all the work to pull things out of tree, and the general movement towards adding new APIs outside of the core API group, I think it's plausible that one day, we may get to a place where we could consider it, but I don't think it's likely for some time, if ever. @tpepper, @jberkus, @LiGgit, and @dims among others may have thoughts here. :) |
Beta Was this translation helpful? Give feedback.
-
|
@youngnick you summed it up. Having 2 years of support for a specific API snapshot is unrealistic right now for all sorts of reasons, and it wasn't even clear that it was what people actually wanted. |
Beta Was this translation helpful? Give feedback.
-
|
All: I'm going to research what actual feature trajectory looks like through Kubernetes, because @johnbelamaric has identified that as a critical question. Stats to come. |
Beta Was this translation helpful? Give feedback.
-
|
@jberkus Any updates on the stats? :) |
Beta Was this translation helpful? Give feedback.
-
|
Nope, got bogged down with other issues, and the question of "what is a feature" in Kubernetes turns out to be a hard one to answer. We don't actually track features outside of a single release cycle; we track KEPs, which can either be part of a feature or the parent of several features, but don't match up 1:1 as features. So first I need to invent a way to identify "features" in a way that works for multiple release cycles. |
Beta Was this translation helpful? Give feedback.
-
|
I was on leave for the past two weeks so chiming in a little late... I am +1 to 3 releases per year. While some folks in the thread note that this increases the heft/risk of each release, I actually think less releases will reduce risk. I'm speaking from an operations perspective as opposed to a development perspective. The current Kubernetes release cadence is so high that most organizations cannot keep up with making a major version update every 3 months regularly, or going out of security support in less than a year. While in theory, releasing more frequently reduces the churn and risk of each release, this is only true if end users are actually able to apply the upgrades. In my experience, this is very challenging and I have not yet seen any organization consistently keep up with the 3 month major upgrade pace for production clusters, especially at a large scale. So, what effectively happens is that end users upgrade less frequently than 3 months, but since that isn't supported, they end up in the situation where they are required to jump multiple major releases at once, which effectively results in much higher risk. 4 vs. 3 releases is >30% more release work, but I do not believe it provides benefit proportional to that work, nor does a quarterly major release cadence match the vast majority of operation teams' upgrade cycles. |
Beta Was this translation helpful? Give feedback.
-
|
(I've converted this to a GitHub discussion, so you can now thread in on specific comments here 🙃) |
Beta Was this translation helpful? Give feedback.
-
|
Sorry this has been forever, but answering the question of "how fast do our features advance" turns out to be really hard, because there is literally no object called a "feature" that persists reliably through multiple releases. To reduce the scope of the problem, I decided to limit this to tracked Enhancements introduced as alpha in 1.12 or 1.13, which were particularly fruitful releases for new features. Limiting it to Tracked kind of limits it to larger features, but I think these are the only ones required to go through alpha/beta/stable anyway (yes/no?). So, in 1.12 and 1.13: 20 new enhancements were introduced Our median number of releases for an enhancement to progress is: Alpha to Beta: 2 releases Given this, it does not look like moving to 3 releases a year would slow down feature development due to the alpha/beta/stable progression requirements. I will note, however, that for many enhancements nagging by the Release Team during each release cycle did provide a goad to resume stalled development. So I'm at least a little concerned that if we make 3/year and don't change how we monitor feature progression at all, it will still slow things down because devs are being nagged less often. |
Beta Was this translation helpful? Give feedback.
-
|
I am a little less worried about the "nag factor" now that we've moved to push-driven enhancements this release (SIG leads track, enhancements team accepts vs. enhancements team tracks). |
Beta Was this translation helpful? Give feedback.
-
|
I'm very worried about it, see new thread. |
Beta Was this translation helpful? Give feedback.
-
|
I am more concerned about the "nag factor" due to the move to push-driven
development; I think the lack of nagging will slow some features down.
However, there are new things at play that will help with it - namely, the
"no more permabeta" where things can't linger in beta forever because their
API gets turned off automatically. At least for things with APIs.
I strongly believe our alpha-to-stable latency, already quite high, will
get worse with 3 releases per year. But ultimately, it's up to the feature
authors. If they want it to go fast enough, they'll have to push for it
more. Missing a train will have higher cost.
Anyway, I personally am, as I said before, ambivalent on this decision.
Putting on my GKE hat, it makes my life easier. Putting on my OSS hat, I
have concerns but nothing that would make me strongly oppose it. It's not a
change I would push for, but I think it's reasonable to see what happens if
we try it.
…On Thu, Feb 4, 2021 at 10:25 AM Elana Hashman ***@***.***> wrote:
I am a little less worried about the "nag factor" now that we've moved to
push-driven enhancements this release (SIG leads track, enhancements team
accepts vs. enhancements team tracks).
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#1290 (reply in thread)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/ACIHRM5VHVO2ID673SBHBBDS5LRBLANCNFSM4U3ONAZQ>
.
|
Beta Was this translation helpful? Give feedback.
-
|
And thank you Josh for the analysis. The median 6 releases goes from 18
months to 24 months, which is not great but also something that is not
forced on feature authors - they could push and get it done in less time if
they need to. It's a rare feature that was making it in the 9 months
before, so it would be a rare feature that is forced to have a longer cycle
than they would have otherwise.
On Thu, Feb 4, 2021 at 10:44 AM John Belamaric <jbelamaric@google.com>
wrote:
… I am more concerned about the "nag factor" due to the move to push-driven
development; I think the lack of nagging will slow some features down.
However, there are new things at play that will help with it - namely, the
"no more permabeta" where things can't linger in beta forever because their
API gets turned off automatically. At least for things with APIs.
I strongly believe our alpha-to-stable latency, already quite high, will
get worse with 3 releases per year. But ultimately, it's up to the feature
authors. If they want it to go fast enough, they'll have to push for it
more. Missing a train will have higher cost.
Anyway, I personally am, as I said before, ambivalent on this decision.
Putting on my GKE hat, it makes my life easier. Putting on my OSS hat, I
have concerns but nothing that would make me strongly oppose it. It's not a
change I would push for, but I think it's reasonable to see what happens if
we try it.
On Thu, Feb 4, 2021 at 10:25 AM Elana Hashman ***@***.***>
wrote:
> I am a little less worried about the "nag factor" now that we've moved to
> push-driven enhancements this release (SIG leads track, enhancements team
> accepts vs. enhancements team tracks).
>
> —
> You are receiving this because you were mentioned.
> Reply to this email directly, view it on GitHub
> <#1290 (reply in thread)>,
> or unsubscribe
> <https://github.com/notifications/unsubscribe-auth/ACIHRM5VHVO2ID673SBHBBDS5LRBLANCNFSM4U3ONAZQ>
> .
>
|
Beta Was this translation helpful? Give feedback.
-
|
I'm going to start a new thread on reminder factors, because these are a bigger deal than I think folks want to address. There are two areas in the project, currently, that are almost entirely dependent on release team reminders (herafter RTR) for development:
With the current 3-month cycle, this means that for 1 month of every cycle RTR doesn't happen, and as a result, these two activities don't happen. This is an extremely broken system. Contributors should not be dependent on personal reminders of any sort to take care of these project priorities, and the project as a whole shouldn't be depending on the RT for them except during Code Freeze. A 4-month cycle will make this problem worse, because we'll be looking a 2 months of every cycle where RTR won't happen, a doubling of the amount of time per year for tests to fail and alpha features to be forgotten. I am not saying that this is a reason NOT to do a 4-month cycle. I am saying that switching to a 4-month cycle makes fixing our broken RTR-based system an urgent priority. Fixing failing tests needs to happen year round. Reviewing features for promotion needs to happen at every SIG meeting. (FWIW, this is an issue that every large OSS project faces, which is why Code Freeze is to awful in so many projects) |
Beta Was this translation helpful? Give feedback.
-
|
Hey everyone, thanks for all of the thoughtful discussion. (Locking this discussion.) |
Beta Was this translation helpful? Give feedback.
and others
-
What would you like to be added:
We should formally discuss whether or not it's a good idea to modify the kubernetes/kubernetes release cadence.
Why is this needed:
The extended release schedule for 1.19 will result in only three minor Kubernetes releases for 2020.
As a result, we've received several questions across a variety of platforms and DMs about whether the project is intending to only have three minor releases/year.
In an extremely scientific fashion, I took this question to a Twitter poll to get some initial feedback: https://twitter.com/stephenaugustus/status/1305902993095774210?s=20
Of the 709 votes, 59.1% preferred three releases over our current, non-2020 target of four.
There's quite a bit of feedback to distill from that thread, so let's start aggregating opinions here.
Strictly my personal opinion:
I'd prefer three releases/year.
@kubernetes/sig-release @kubernetes/sig-architecture @kubernetes/sig-testing
/assign
/milestone v1.20
/priority important-longterm
Beta Was this translation helpful? Give feedback.
All reactions