LACP support: added lacprate and xmitpolicy

[killianmuldoon]

No description provided.

[killianmuldoon]

Closed as stale.

[kbowlingns]

@killianmuldoon does this require design changes? I would like to be able to set lacprate

[killianmuldoon]

@kbowlingns I don't think this would require major changes beyond some clean up - this was done as a proof of concept though so I'd need to take another look. Have you tried pulling this patch to see if it meets your needs for setting lacprate?

[howels]

These changes look useful for using the bond CNI in a real-world network where things like link balancing need to be optimised. Any chance this could be reviewed?

[killianmuldoon]

@howels I've reopened this - I think we could move toward reviewing and merging it. Can you test to see if it meets your requirements?

[howels]

@killianmuldoon can you verify my config? I've built your branch but see some error adding the bond to the container:

  Normal   AddedInterface          2m12s                  multus             Add net1 [] from kube-system/sriov-pf1
  Normal   AddedInterface          2m12s                  multus             Add eth0 [10.42.134.70/32]
  Normal   AddedInterface          2m11s                  multus             Add net2 [] from kube-system/sriov-pf2
  Warning  FailedCreatePodSandBox  2m11s                  kubelet            Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "1bcb18069dbbf63de91b2e0cdb3db5b273094c2ecf04f2f108259e16edbd1c34": [default/iperf-clients-cqgc7:bond-net1]: error adding container to network "bond-net1": Failed to create bonded link (bond0), error: Failed to add link (bond0) to the netNsHandle, error: invalid argument
  Normal   AddedInterface          2m10s                  multus             Add eth0 [10.42.134.75/32]
  Normal   AddedInterface          2m10s                  multus             Add net1 [] from kube-system/sriov-pf1

This is using the following NAD:

apiVersion: "k8s.cni.cncf.io/v1"
kind: NetworkAttachmentDefinition
metadata:
  name: bond-net1
spec:
  config: '{
    "type": "bond",
    "cniVersion": "0.3.1",
    "name": "bond-net1",
    "ifname": "bond0",
    "mode": "802.3ad",
    "xmit_hash_policy": "layer3+4",
    "failOverMac": 1,
    "linksInContainer": true,
    "miimon": "100",
    "links": [
      {"name": "net1"},
      {"name": "net2"}
    ],
    "ipam": {
      "type": "host-local",
      "subnet": "192.168.3.0/24",
      "rangeStart": "192.168.3.200",
      "rangeEnd": "192.168.3.216",
      "routes": [{
        "dst": "0.0.0.0/0"
      }]
    }
  }'

I am able to allocate the component SR-IOV VFs as NICs and use those so not sure what is failing:

      annotations:
        k8s.v1.cni.cncf.io/networks: '[
          {"name": "sriov-pf1",
          "namespace": "kube-system",
          "interface": "net1"},
          {"name": "sriov-pf2",
          "namespace": "kube-system",
          "interface": "net2"},
          {"name": "bond-net1",
          "namespace": "kube-system",
          "interface": "bond0"}
          ]'
 #       v1.multus-cni.io/default-network: bond-net1
 ......
         resources:
          requests:
            intel.com/intel_sriov_PF_1: '1'
            intel.com/intel_sriov_PF_2: '1'
          limits:
            intel.com/intel_sriov_PF_1: '1'
            intel.com/intel_sriov_PF_2: '1'

Is there something wrong in that config? I built /opt/cni/bin/bond from 2e41fe1

[killianmuldoon]

@howels I think it should be xmitHashPolicy rather than xmit_hash_policy in your config. There's an example config here.

[killianmuldoon]

@howels have you assessed this to see if it meets your use case? If so I can work on getting a final review and merge through.

[howels]

@killianmuldoon sorry, this was introducing too much complexity so we backed away from SR-IOV devices and used shared RDMA with the bond in the host OS instead.

[killianmuldoon]

@howels I'm going to close this as stale again until we can get together some documented testable use cases. Thanks for taking another look at this!

[sainath40]

Hi @killianmuldoon , I am kind of having similar usecase in my environment. Where i have 8 virtual functions in a pod, i would like to aggregate them to a one single bond interface using mode 802.3ad. can it be possible with bond-cni ?

[killianmuldoon]

Hi @killianmuldoon , I am kind of having similar usecase in my environment. Where i have 8 virtual functions in a pod, i would like to aggregate them to a one single bond interface using mode 802.3ad. can it be possible with bond-cni ?

@sainath40 I'm not currently working on bond CNI I'm afraid. It might be a good idea to open an issue in the repo describing your use case.