Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Path normalization breaks decoding data classes inside maps and sets incorrect map keys for property sources #443

Closed
rocketraman opened this issue Aug 16, 2024 · 6 comments · Fixed by #444

Comments

@rocketraman
Copy link
Contributor

rocketraman commented Aug 16, 2024

The implementations of AbstractUnnormalizedKeysDecoders which was created in PR #413 "restore" the unnormalized node keys.

Unfortunately, when a decoder that requires normalization is further down in the tree, such as a data class decoder, it sees the unnormalized values and can therefore fail to read the config correctly.

The solution is to be less heavy-handed about de-normalization for map decoders and the Hikara data source decoder, and have them internally decode nodes only when they need to.

Additionally, the sourceKey was being set incorrectly to null for props-based sources like the CommandLinePropertySource.

@rocketraman
Copy link
Contributor Author

@sksamuel Would appreciate a review and merge of this asap, because current master is a bit broken.

@osoykan
Copy link
Contributor

osoykan commented Sep 17, 2024

I think that with the new version 2.8.0, another problem was introduced in the same area. Map keys from CommandLine was fixed but, now, if it comes from environment variables, the key ends up being lowercasekey. I didn't have the time to write a test case but, just saw on the CI, the behaviour was affected.

@rocketraman
Copy link
Contributor Author

rocketraman commented Sep 17, 2024

I think that with the new version 2.8.0, another problem was introduced in the same area. Map keys from CommandLine was fixed but, now, if it comes from environment variables, the key ends up being lowercasekey. I didn't have the time to write a test case but, just saw on the CI, the behaviour was affected.

@osoykan The key is lowercase because all keys are normalized to lowercase now due to the addition of the path normalizer by default, but it should still set the config correctly. Does it?

Also FYI, there is possibly related issue #410 which is fixed by PR #414.

@osoykan
Copy link
Contributor

osoykan commented Sep 17, 2024

@rocketraman I've reproduced the case in the commit: https://github.com/osoykan/hoplite/commit/d858187a057f58e9066ef0ca9b63ff2fedbe07cc

When the key is lowercase, then my access strategy needs to be changed, too, which is breaking change.

Could be fixed with your MR, indeed, if you could give it a try with the test-case I've added that would be perfect!

@rocketraman
Copy link
Contributor Author

@osoykan Oh that's interesting -- the test is fine if addEnvironmentSource (or addCommandLineSource) are used by themselves, and it is also fine if addResourceOrFileSource is used by itself, but if addResourceOrFileSource is combined with either addEnvironmentSource or addCommandLineSource (or both), it fails.

I will look into it.

@rocketraman
Copy link
Contributor Author

rocketraman commented Sep 17, 2024

@osoykan I created #447, and it should be fixed in #448 .

rocketraman pushed a commit to rocketraman/hoplite that referenced this issue Sep 17, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants