-
Notifications
You must be signed in to change notification settings - Fork 24
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Make yaml.Unmarshal strict #271
Conversation
Codecov Report
@@ Coverage Diff @@
## main #271 +/- ##
=======================================
Coverage 66.83% 66.83%
=======================================
Files 73 73
Lines 4149 4149
=======================================
Hits 2773 2773
Misses 1197 1197
Partials 179 179
Flags with carried forward coverage won't be shown. Click here to find out more.
Help us with your feedback. Take ten seconds to tell us how you rate us. |
@@ -37,6 +37,6 @@ func TestDuration_MarshalYAML(t *testing.T) { | |||
func TestDuration_UnmarshalYAML(t *testing.T) { | |||
expectedMsg := testMessage{Elapsed: Duration{2 * time.Hour}} | |||
var actualMsg testMessage | |||
require.NoError(t, yaml.Unmarshal([]byte("elapsed: 2h\n"), &actualMsg)) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@ronensc What is your general rule when you want to use UnmarshalStrict as opposed to Unmarshal?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Assume we have a type like this:
type Person struct {
Name string `yaml:"name"`
Age int `yaml:"age"`
}
and data that we want to parse:
person:
name: John
Age: 42
Notice that the age field is misspelled (it should be age
with lowercase a).
Using unstrict yaml.Unmarshal()
won't raise an error on this. It will simply ignore the unrecognized field Age
and set the default value for the missing field age
.
I think in our use-case, where we parse config files, we want to make sure that all fields are recognized and emit an error otherwise.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
So aren't there numerous places in the code where we Marshal the config data where we should also use UnmarshalStrict ?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I replaced all the yaml.Unmarshal()
invocations that I found (I found only 3).
Please let me know if I missed something.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There are also lots of json.Unmarshal() calls.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Unfortunately, the json
library doesn't have a strict version of Unmarshal()
like the yaml
has.
But, we can get the strict behavior with json.NewDecoder
and DisallowUnknownFields()
as suggested here:
https://groups.google.com/g/golang-nuts/c/MJuJDte5YHg/m/JMs0S-4uAQAJ
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can't some of those be changed to to yaml.UnmarshalStrict?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
lgtm
Indeed I remember having seen hostname
and hostName
in some places and was confused about that
Note: This PR deals only with |
90e775f
to
3f5d194
Compare
Using
yaml.UnmarshalStrict()
will prevent us from making typos in our yamls without being noticed.