Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

bump netlink&cni&golang version and support add ecmp route #227

Merged
merged 1 commit into from
Nov 6, 2022

Conversation

cyclinder
Copy link
Contributor

@cyclinder cyclinder commented Sep 28, 2022

Support add ECMP default route:

When I add a sriov-cni NIC to the pod by using multus( The first NIC in the pod is assigned by calico ). I want the default route for the second NIC (net1) to be created correctly, but there is actually only one default route for calico.

root@172-17-8-110:/var/lib/cni/sriov# kubectl exec -it sriov-overlay-8475768cfd-r6slw sh
kubectl exec [POD] [COMMAND] is DEPRECATED and will be removed in a future version. Use kubectl exec [POD] -- [COMMAND] instead.
/ # ip r
default via 169.254.1.1 dev eth0
169.254.1.1 dev eth0 scope link
172.17.8.0/21 dev net1 scope link  src 172.17.8.191

This should actually have two default routes:

default via 169.254.1.1 dev eth0
default via 172.17.8.1 dev net1  # this is the default route for net1
169.254.1.1 dev eth0 scope link
172.17.8.0/21 dev net1 scope link  src 172.17.8.191

In the calico + macvlan scenario, where two default routes are created. I checked the source code of sriov-cni and found that the version of CNI it references is too old to cause this problem.

github.com/containernetworking/cni: v0.8.1

github.com/containernetworking/cni: v1.1.1

In summary, I solved this problem by upgrading CNI to v1.1.1, and I tested it in my local environment, It works well. As an add-on, we also have to upgrade the netlink and golang versions.

That's all this PR is doing.

Signed-off-by: cyclinder qifeng.guo@daocloud.io

@zeeke
Copy link
Member

zeeke commented Oct 5, 2022

hi @cyclinder , can you provide a little more context about this change?

@cyclinder
Copy link
Contributor Author

Hi @zeeke , I have just updated the context of this change, please review it. Thanks!

@cyclinder cyclinder changed the title bump netlink&cni&golang version and fix failed to ecmp route bump netlink&cni&golang version and support add ecmp route Oct 7, 2022
@cyclinder
Copy link
Contributor Author

I think this change makes sense :) , could it be merged into the master branch if appropriate?

@zeeke
Copy link
Member

zeeke commented Oct 7, 2022

@cyclinder thanks for explaining.

nit: is the bump to go1.19 strictly needed? if not, can you at least split it in a different commit?

other than that,
LGTM

Thanks

@cyclinder
Copy link
Contributor Author

cyclinder commented Oct 8, 2022

@cyclinder thanks for explaining.

nit: is the bump to go1.19 strictly needed? if not, can you at least split it in a different commit?

other than that, LGTM

Thanks

@zeeke Yeah, bumping to go1.19 is strictly needed. and I could raise another PR to bump to go1.19,After waiting for #PR 228 to merge to master brach, This PR is ready to merged, WDYT?

@coveralls
Copy link

coveralls commented Oct 10, 2022

Pull Request Test Coverage Report for Build 3204643668

Warning: This coverage report may be inaccurate.

This pull request's base commit is no longer the HEAD commit of its target branch. This means it includes changes from outside the original pull request, including, potentially, unrelated coverage changes.

Details

  • 0 of 0 changed or added relevant lines in 0 files are covered.
  • No unchanged relevant lines lost coverage.
  • Overall coverage remained the same at 34.347%

Totals Coverage Status
Change from base Build 3015757156: 0.0%
Covered Lines: 339
Relevant Lines: 987

💛 - Coveralls

@zeeke
Copy link
Member

zeeke commented Oct 10, 2022

@Eoghan1232 please, have a look at this.

Eoghan1232
Eoghan1232 previously approved these changes Oct 10, 2022
Copy link
Collaborator

@Eoghan1232 Eoghan1232 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

overall lgtm, once we clean up the pr to be in line with #228

we can look to merge both

go.mod Show resolved Hide resolved
@Eoghan1232 Eoghan1232 self-requested a review October 10, 2022 14:44
@Eoghan1232 Eoghan1232 dismissed their stale review October 10, 2022 14:44

changes needed.

@cyclinder cyclinder force-pushed the fix_ecmp_route branch 2 times, most recently from 5d826e0 to 48a742c Compare October 25, 2022 15:21
@cyclinder cyclinder requested review from e0ne and Eoghan1232 and removed request for Eoghan1232 and e0ne October 25, 2022 15:26
@cyclinder
Copy link
Contributor Author

Hi @Eoghan1232 Can you take a look ? thanks

@Eoghan1232
Copy link
Collaborator

lgtm!
@adrianchiris can you take a look?

Copy link
Contributor

@adrianchiris adrianchiris left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Greeting !

while i have no objections with the changes introduced. its a bit unclear to me what fixed ecmp route issue you described.

also updating commit message with a bit more info on the fix would be great !

go.mod Show resolved Hide resolved
go.mod Outdated
github.com/onsi/ginkgo v1.16.4
github.com/onsi/gomega v1.17.0
github.com/stretchr/testify v1.5.1
github.com/vishvananda/netlink v1.2.0-beta
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is netlink version bump needed ? if so do you know what was added/fixed ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, In the latest release of netlink, they have added the RouteAddEcmp function, which is the most important part.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK here is what i found:

support for calling RouteAddEcmp was added to containernetworking/plugins in v1.0.0 in this commit

in sriov-cni code we call ConfigureInterface from containernetworking plugins ipam package.

so we need to bump containernetworking/plugins verison.

in v1.1.1 an older version of netlink is used
which apparently is good enough, as bumping netlink version here will not affect the netlink version set in plugins go.mod (we are using go modules to build so each package provides its own dependencies)

still, i do not think netlink version should be bumped as there is no reason to bump it.
am i missing anything ?

Copy link
Contributor

@adrianchiris adrianchiris Nov 6, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ah i see latest PR update now uses netlink v1.1.1 and not 1.2.0 so disregard my comment RE netlink version :)

update cni/cni-plugins to v1.1.2/v1.1.1, and netlink/ginkgo/gomega are sub-dependencies of cni/cni-plugins,
they will be upgraded together as well.

Signed-off-by: cyclinder <qifeng.guo@daocloud.io>
@adrianchiris adrianchiris merged commit e0344dd into k8snetworkplumbingwg:master Nov 6, 2022
@cyclinder
Copy link
Contributor Author

Thanks @adrianchiris

Do you have plans to publish a new release?

@adrianchiris
Copy link
Contributor

ATM, release is on an "on-demand" basis. can you open an issue requesting a new release ?

@cyclinder
Copy link
Contributor Author

Yes! I could open an issue for this, I think it's meaningful.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants