Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

flannel: allow input ipam parameters as basis for delegate #532

Merged
merged 1 commit into from
Oct 7, 2020

Conversation

dverbeir
Copy link
Contributor

@dverbeir dverbeir commented Sep 10, 2020

(NOTE: This work started under PR #527 which was a combination of this and the addition of logging; now split into 2 PRs)

This change allows providing an 'ipam' section as part of the input network configuration for flannel. It is then used as basis to construct the ipam parameters provided to the delegate.

All parameters from the input ipam are preserved except:

  • 'subnet' which is set to the flannel host subnet
  • 'routes' which is complemented by a route to the flannel network.

One use case of this feature is to allow adding back the routes to the cluster services and/or to the hosts (HostPort) when using isDefaultGateway=false. In that case, the bridge plugin does not install a default route and, as a result, only pod-to-pod connectivity would be available.

Example:

  {
    "name": "cbr0",
    "cniVersion": "0.3.1",
    "type": "flannel",
    "ipam": {
      "routes": [
        {
          "dst": "192.168.242.0/24"
        },
        {
          "dst": "10.96.0.0/12"
        }
      ],
      "unknown-param": "value"
    },
    "delegate": {
      "hairpinMode": true,
      "isDefaultGateway": false
    }
    ...
  }

This results in the following 'ipam' being provided to the delegate:

    {
      "routes" : [
        {
          "dst": "192.168.242.0/24"
        },
        {
          "dst": "10.96.0.0/12"
        },
        {
          "dst" : "10.1.0.0/16"
        }
      ],
      "subnet" : "10.1.17.0/24",
      "type" : "host-local"
      "unknown-param": "value"
    }

where "10.1.0.0/16" is the flannel network and "10.1.17.0/24" is the host flannel subnet.

Note that this also allows setting a different ipam 'type' than "host-local".

Copy link
Contributor

@bboreham bboreham left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems broadly OK but I have one question.

Comment on lines +49 to +50
// IPAM field "replaces" that of types.NetConf which is incomplete
IPAM map[string]interface{} `json:"ipam,omitempty"`
Copy link
Contributor

@bboreham bboreham Sep 16, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why use a string map rather than a struct ?
(It came as a surprise to me that containernetworking/cni does not define any useful IPAM struct, but even so, why don't you?)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Even though my own use case is about the routes, I thought it would be nice to allow transparently passing any IPAM parameter that could apply to the delegate (bridge or other). In order not to limit the possibilities there, a string map looked appropriate to me (I'm not aware of any way to unmarshal into a struct while preserving and giving access to all extra fields not defined in the struct).
You will see in flannel_linux.go:getDelegateIPAM() that the incoming string map, when present, is used as basis for the delegate ipam.

@dcbw
Copy link
Member

dcbw commented Sep 23, 2020

@rajatchopra does this look sane to you? Would you use an override capability for flannel IPAM?

This change allows providing an 'ipam' section as part of the
input network configuration for flannel. It is then used as
basis to construct the ipam parameters provided to the delegate.

All parameters from the input ipam are preserved except:
* 'subnet' which is set to the flannel host subnet
* 'routes' which is complemented by a route to the flannel
  network.

One use case of this feature is to allow adding back the routes
to the cluster services and/or to the hosts (HostPort) when
using isDefaultGateway=false. In that case, the bridge plugin
does not install a default route and, as a result, only pod-to-pod
connectivity would be available.

Example:
    {
      "name": "cbr0",
      "cniVersion": "0.3.1",
      "type": "flannel",
      "ipam": {
        "routes": [
          {
            "dst": "192.168.242.0/24"
          },
          {
            "dst": "10.96.0.0/12"
          }
        ],
        "unknown-param": "value"
      },
      "delegate": {
        "hairpinMode": true,
        "isDefaultGateway": false
      }
      ...
    }

This results in the following 'ipam' being provided to the delegate:
    {
      "routes" : [
        {
          "dst": "192.168.242.0/24"
        },
        {
          "dst": "10.96.0.0/12"
        },
        {
          "dst" : "10.1.0.0/16"
        }
      ],
      "subnet" : "10.1.17.0/24",
      "type" : "host-local"
      "unknown-param": "value"
    }

where "10.1.0.0/16" is the flannel network and "10.1.17.0/24" is
the host flannel subnet.

Note that this also allows setting a different ipam 'type' than
"host-local".

Signed-off-by: David Verbeiren <david.verbeiren@tessares.net>
@dverbeir
Copy link
Contributor Author

I re-based on the latest master so the tests pass now.

How to move on with this change? Do we need the feedback from @rajatchopra?
Due to the fairly transparent way in which it's implemented, it could prove useful for various cases.

Copy link
Contributor

@bboreham bboreham left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@squeed
Copy link
Member

squeed commented Oct 7, 2020

seems fine

@squeed squeed merged commit 6df03d7 into containernetworking:master Oct 7, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants