You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
In #1109 a new set of test files was added that continues the use of an established naming convention! However, the set of message tags checked in those files is also the set of tags we need to keep stable because other people are relying on them. If we add new tests with this naming convention (and do nothing else), we might believe those tags are also covered by the stability guarantee, and that's an unnecessary maintenance burden.
I guess it would be a good idea to create a list in zonemaster/zonemaster that lists all the tags that we need to keep stable. But that's not enough. It should be visible somehow in the test files which tags are stable and which aren't. Otherwise it's really easy to forget that some tags are stable when making and reviewing updates. Maybe we could just rename the old Test-XXXX-X.t to Stable-XXXX-X.t? Maybe there are better ideas?
The text was updated successfully, but these errors were encountered:
For some time we've had the policy to keep the exact semantics of some of the messages stable when updating Zonemaster. However with such a stability guarantee there is a risk that the test case specifications get more complex than they would otherwise have to be. It's been decided that we should not make any extra effort to preserve the exact semantics of messages between versions of Zonemaster.
In #1109 a new set of test files was added that continues the use of an established naming convention! However, the set of message tags checked in those files is also the set of tags we need to keep stable because other people are relying on them. If we add new tests with this naming convention (and do nothing else), we might believe those tags are also covered by the stability guarantee, and that's an unnecessary maintenance burden.
I guess it would be a good idea to create a list in zonemaster/zonemaster that lists all the tags that we need to keep stable. But that's not enough. It should be visible somehow in the test files which tags are stable and which aren't. Otherwise it's really easy to forget that some tags are stable when making and reviewing updates. Maybe we could just rename the old Test-XXXX-X.t to Stable-XXXX-X.t? Maybe there are better ideas?
The text was updated successfully, but these errors were encountered: