Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(*): avoid setting nil for missing fields #463

Closed
wants to merge 1 commit into from

Conversation

samugi
Copy link
Member

@samugi samugi commented Aug 19, 2024

when filling defaults, the existing logic sets nil on missing fields, but in Kong Gateway explicit nils result in the corresponding fields to be unset, causing problems such as: KAG-5157

This change removes any unset fields that don't have a default from the configuration instead of explicitly setting them to nil.

@reviewers: could this be a breaking change? I tested it in decK and it seems to work but I'm worried about other projects using this lib. An alternative would be creating another utils function with the new behavior (which I think would require updating at least deck, go-database-reconciler and probably kubernetes-ingress-controller), but if we're fairly confident the func is only used while interacting with decK then it might be considered safe enough.

Searching occurrences.

For more details see: KAG-5210

@samugi samugi requested review from a team as code owners August 19, 2024 15:44
@samugi samugi marked this pull request as draft August 19, 2024 15:44
@samugi samugi force-pushed the fix/avoid-setting-nil-for-missing-fields branch 3 times, most recently from 4fe0227 to 0492afb Compare August 21, 2024 16:39
@codecov-commenter
Copy link

codecov-commenter commented Aug 21, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 59.94%. Comparing base (c64125c) to head (6dbf5df).

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #463      +/-   ##
==========================================
+ Coverage   59.85%   59.94%   +0.08%     
==========================================
  Files          71       71              
  Lines        4449     4449              
==========================================
+ Hits         2663     2667       +4     
+ Misses       1167     1165       -2     
+ Partials      619      617       -2     
Flag Coverage Δ
2.1 36.36% <100.00%> (+0.08%) ⬆️
2.2 48.77% <100.00%> (+0.08%) ⬆️
2.3 49.40% <100.00%> (+0.08%) ⬆️
2.4 49.44% <100.00%> (+0.08%) ⬆️
2.5 49.44% <100.00%> (+0.08%) ⬆️
2.6 49.44% <100.00%> (+0.08%) ⬆️
2.7 51.13% <100.00%> (+0.08%) ⬆️
2.8 51.13% <100.00%> (+0.08%) ⬆️
3.0 55.27% <100.00%> (+0.08%) ⬆️
3.1 56.88% <100.00%> (+0.08%) ⬆️
3.2 56.88% <100.00%> (+0.08%) ⬆️
3.3 56.88% <100.00%> (+0.08%) ⬆️
3.4 59.24% <100.00%> (+0.08%) ⬆️
3.5 57.06% <100.00%> (+0.08%) ⬆️
3.6 57.06% <100.00%> (+0.08%) ⬆️
community 44.14% <100.00%> (+0.08%) ⬆️
enterprise 58.37% <100.00%> (+0.08%) ⬆️
integration 59.94% <100.00%> (+0.08%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Comment on lines +396 to +397
// if no default exists, remove the field
delete(res, fname)
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is the actual change

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are some plugins that require explicit null here.

We need to allow users to explicitly specify a null value, but agree that we should not populate null from defaults

@samugi samugi marked this pull request as ready for review August 21, 2024 20:46
@samugi
Copy link
Member Author

samugi commented Aug 21, 2024

not sure why the linter got mad

@randmonkey
Copy link
Contributor

not sure why the linter got mad

https://github.com/Kong/go-kong/actions/runs/10493904848/job/29068947573?pr=463
Looks like the linter was updated some day but it did not notify us

@pmalek
Copy link
Member

pmalek commented Aug 22, 2024

@samugi #464 This should solve the linter issues and align linter version to be the same on CI and locallly

@bungle
Copy link
Member

bungle commented Aug 22, 2024

For visibility:

I do not fully understand why there is even this FillPluginsDefaults. I think it is wrong that the defaults or nulls are filled to config (prior sending them to Kong API). If user has .yaml file, the tool should not invent any fields in it, and send only those that user has specified, e.g. even when user has specified field: ~ (aka null), that should be sent to kong, but nothing else. Kong, at least the Lua CP will fill the defaults, and do proper thing.

Now if tool itself needs these to be filled (e.g. for easier diffing or something), it is another question, but that should not affect what is sent to wire to Kong API (provided that Koko API works the same).

This of course means that the tool should also never use PATCH or POST, just PUT and DELETE and GET. And if it needs for this to invent anything, I would say only the id but that you need to take some care with. E.g. multiple runs should probably produce same id (or primary key).

That said, we can first stop filling nulls (as it fixes the immediate issue of deprecated shorthand precedence), and perhaps later (in other PR) stop filling defaults.

@mheap
Copy link
Member

mheap commented Aug 22, 2024

That said, we can first stop filling nulls (as it fixes the immediate issue of deprecated shorthand precedence), and perhaps later (in other PR) stop filling defaults.

Stop filling default nulls, you mean? Active null should still be accepted

@samugi
Copy link
Member Author

samugi commented Aug 22, 2024

That said, we can first stop filling nulls (as it fixes the immediate issue of deprecated shorthand precedence), and perhaps later (in other PR) stop filling defaults.

Stop filling default nulls, you mean? Active null should still be accepted

yes @mheap that is correct, this change is intended to only apply to auto-filled defaults

upon filling defaults for the configuration, the existing logic was
setting `nil` on missing fields, but in Kong Gateway, explicit `nil`
are not treated as a synonym of "missing field".
This change removes any missing fields from the configuration instead of
explicitly setting such fields as `nil`.
@samugi samugi force-pushed the fix/avoid-setting-nil-for-missing-fields branch from 0492afb to 6dbf5df Compare August 22, 2024 11:49
@samugi
Copy link
Member Author

samugi commented Aug 22, 2024

Note: this appears to be breaking the decK diff output (where now all nulls show up as diffs) - probably expected given the change here, because of this, so we might need to do it differently (?), e.g. like @bungle says, by preventing default to be filled at all when the config is sent to Kong (and keep the behavior unchanged for diffs). I suspect that change must be done in decK somewhere else instead of here.

@samugi samugi marked this pull request as draft August 22, 2024 12:14
@samugi
Copy link
Member Author

samugi commented Aug 24, 2024

second attempt: Kong/go-database-reconciler#133

@samugi samugi closed this Sep 3, 2024
@samugi samugi deleted the fix/avoid-setting-nil-for-missing-fields branch September 3, 2024 13:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants