Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ensure iptables chain creation is idempotent #408

Merged
merged 1 commit into from
Nov 13, 2019

Conversation

tgross
Copy link
Contributor

@tgross tgross commented Nov 4, 2019

Concurrent use of the portmap and firewall plugins can result in errors during iptables chain creation:

  • The portmap plugin has a time-of-check-time-of-use race where it checks for existence of the chain but the operation isn't atomic.
  • The firewall plugin doesn't check for existing chains and just returns an error.

This commit makes both operations idempotent by creating the chain and then discarding the error if it's caused by the chain already existing. I've added a test for idempotent creation in the firewall plugin but the portmap plugin already had one so I've left that in place.

@tgross
Copy link
Contributor Author

tgross commented Nov 4, 2019

While it wasn't the intent of this PR, this may fix #370, which is bubbling up a similar error condition as we saw in hashicorp/nomad#6567

plugins/meta/portmap/chain.go Outdated Show resolved Hide resolved
plugins/meta/portmap/chain.go Show resolved Hide resolved
@@ -43,7 +43,16 @@ func ensureChain(ipt *iptables.IPTables, table, chain string) error {
}
}

return ipt.NewChain(table, chain)
err = ipt.NewChain(table, chain)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM for your intention and implementation, but IMO, idempotency should be done in iptables package because decoupling is important for us... cc @squeed waiting for your suggestion...

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It could be done there, sure. But the iptables package is a fairly bare-bones wrapper around the command line tools. They also don't have any documentation promising idempotency, whereas containernetworking/plugins does.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well, I know the maintainer :-).

@tgross tgross force-pushed the idempotent_chain_creation branch from 163447f to 3d0ddef Compare November 6, 2019 15:08
@tgross tgross force-pushed the idempotent_chain_creation branch 2 times, most recently from 0d001e3 to 81426da Compare November 6, 2019 19:37
@tgross
Copy link
Contributor Author

tgross commented Nov 6, 2019

I think at this point I've resolved the items from code review, but I think there's still the open question to @squeed about whether this should be handled upstream in go-iptables instead.

pkg/utils/iptables.go Show resolved Hide resolved
pkg/utils/iptables_test.go Show resolved Hide resolved
plugins/meta/firewall/iptables.go Outdated Show resolved Hide resolved
pkg/utils/iptables.go Show resolved Hide resolved
Concurrent use of the `portmap` and `firewall` plugins can result in
errors during iptables chain creation:

- The `portmap` plugin has a time-of-check-time-of-use race where it
  checks for existence of the chain but the operation isn't atomic.
- The `firewall` plugin doesn't check for existing chains and just
  returns an error.

This commit makes both operations idempotent by creating the chain and
then discarding the error if it's caused by the chain already
existing. It also factors the chain creation out into `pkg/utils` as a
site for future refactoring work.

Signed-off-by: Tim Gross <tim@0x74696d.com>
@tgross tgross force-pushed the idempotent_chain_creation branch from 81426da to 58dd90b Compare November 11, 2019 15:00
Copy link
Member

@mars1024 mars1024 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm, thanks for your patience!

@dcbw
Copy link
Member

dcbw commented Nov 13, 2019

/lgtm

@squeed
Copy link
Member

squeed commented Nov 13, 2019

lgtm!

@squeed squeed merged commit 497560f into containernetworking:master Nov 13, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants