apps.vlan: improve performance, configurable encapsulation

The performance of the VlanMux app (apps.vlan.vlan) is improved by

  - avoiding use of pairs()
  - avoiding string composition
  - serialisation of loops to avoid branches

The forwarding strategy during de-multiplexing has been changed to
forward all packets from the trunk links and accepting drops on vlan
links which are full (instead of only forwarding as many packets as
the fullest vlan link can accept).

Support for different encapsulation schemes has been added: dot1q,
dot1ad and an explicit ethertype to cover non-standard encapsulations.

src/apps/vlan/README.md

@@ -6,49 +6,83 @@ can multiplex and demultiplex packets to different output ports based on tag.
 
 ## Tagger (apps.vlan.vlan)
 
-The `Tagger` app adds a VLAN tag, with the configured value, to packets
-received on its `input` port and transmits them on its `output` port.
+The `Tagger` app adds a VLAN tag, with the configured value and
+encapsulation, to packets received on its `input` port and transmits
+them on its `output` port.
 
 ### Configuration
 
-—  Key **tag**
+—  Key **encapsulation**
+
+*Optional*. The Ethertype to use as encapsulation for the
+VLAN. Permitted values are the strings _"dot1q"_ and _"dot1ad"_ or a
+number to select an arbitrary Ethertype.  _"dot1q"_ and _"dot1ad"_
+correspond to the Ethertypes 0x8100 and 0x88a8, respectively,
+according to the IEEE standards 802.1Q and 802.1ad.
 
-*Required*. VLAN tag to add or remove from the packet.
+If a number is given, it is truncated to 16 bits.  This feature is
+intended to allow interoperation with vendors that do not use one of
+the standard encapsulations (a prominent example being the value
+0x9100, which can still be found in practice for double-tagging
+instead of 0x88a8).
 
+The default is _"dot1q"_.
+
+—  Key **tag**
+
+*Required*. VLAN tag to add or remove from the packet.  The value must
+be a number in the range 1-4094 (inclusive).
 
 ## Untagger (apps.vlan.vlan)
 
-The `Untagger` app checks packets received on its `input` port for a VLAN tag,
-removes it if it matches with the configured VLAN tag and transmits them on its
-`output` port. Packets with other VLAN tags than the configured tag will be
-dropped.
+The `Untagger` app checks packets received on its `input` port for a
+VLAN tag, removes it if it matches with the configured VLAN tag and
+transmits them on its `output` port. Packets with other VLAN tags than
+the configured tag are dropped.
 
 ### Configuration
 
-—  Key **tag**
+—  Key **encapsulation**
+
+*Optional*. See above.
 
-*Required*. VLAN tag to add or remove from the packet.
+—  Key **tag**
 
+*Required*. VLAN tag to add or remove from the packet.  The value must
+be a number in the range 1-4094 (inclusive).
 
 ## VlanMux (apps.vlan.vlan)
 
-Despite the name, the `VlanMux` app can act both as a multiplexer, i.e. receive
-packets from multiple different input ports, add a VLAN tag and transmit them
-out onto one, as well as receiving packets from its `trunk` port and
-demultiplex it over many output ports based on the VLAN tag of the received
-packet.
+Despite the name, the `VlanMux` app can act both as a multiplexer,
+i.e. receive packets from multiple different input ports, add a VLAN
+tag and transmit them out onto one, as well as receiving packets from
+its `trunk` port and demultiplex it over many output ports based on
+the VLAN tag of the received packet.  It supports the notion of a
+"native VLAN" by mapping untagged frames on the trunk port to a
+dedicated output port.
+
+A packet received on its `trunk` input port must either be untagged or
+tagged with the encapsulation as specified with the **encapsulation**
+configuration option.  Otherwise, the packet is dropped.
+
+If the Ethernet frame is tagged, the VLAN ID is extracted and the
+packet is transmitted on the port named `vlan<vid>`, where `<vid>` is
+the decimal representation of the VLAN ID.  If no such port exists,
+the packet is dropped.
+
+If the Ethernet frame is untagged, it is transmitted on the port named
+`native` or dropped if no such port exists.
 
-Packets received on its `trunk` input port with Ethernet type 0x8100 are
-inspected for the VLAN tag and transmitted on an output port `vlanX` where *X*
-is the VLAN tag parsed from the packet. If no such output port exists the
-packet is dropped. Received packets with an Ethernet type other than 0x8100 are
-transmitted on its `native` output port,
+A packet received on a port named `vlan<vid>` is tagged with the VLAN
+ID `<vid>` according to the configured encapsulation and transmitted
+on the trunk port.
+
+A packet received on the port named `native` is transmitted as is on
+the trunk port.
+
+### Configuration
 
-Packets received on its `native` input port are transmitted verbatim on its
-`trunk` output port.
+—  Key **encapsulation**
 
-Packets received on input ports named `vlanX`, where *X* is a VLAN tag, will
-have the VLAN tag *X* added and then be transmitted on its `trunk` output port.
+*Optional*. See above.
 
-There is no configuration for the `VlanMux` app, simply connect it to your
-other apps and it will base its actions on the name of the ports.

src/apps/vlan/vlan.lua

@@ -7,36 +7,53 @@ local lib = require("core.lib")
 
 local C = ffi.C
 local receive, transmit = link.receive, link.transmit
+local full, empty = link.full, link.empty
 local cast = ffi.cast
 local htons, htonl = lib.htons, lib.htonl
 local ntohs, ntohl = htons, htonl
 
-Tagger = {}
-Untagger = {}
+local default_encap = { default = "dot1q" }
+Tagger = {
+   config = {
+      encapsulation = default_encap,
+      tag = { required = true }
+   }
+}
+Untagger = {
+   config = {
+      encapsulation = default_encap,
+      tag = { required = true }
+   }
+}
+VlanMux = {
+   config = {
+      encapsulation = default_encap,
+   }
+}
 
--- 802.1q
-local dot1q_tpid = 0x8100
+local tpids = { dot1q = 0x8100, dot1ad = 0x88A8 }
 local o_ethernet_ethertype = 12
 local uint32_ptr_t = ffi.typeof('uint32_t*')
 
 
--- build a VLAN tag consisting of 2 bytes of TPID set to 0x8100 followed by the
--- TCI field which in turns consists of PCP, DEI and VID (VLAN id). Both PCP
--- and DEI is always 0
-local function build_tag(vid)
-   return htonl(bit.bor(bit.lshift(dot1q_tpid, 16), vid))
+-- build a VLAN tag consisting of 2 bytes of TPID followed by the TCI
+-- field which in turns consists of PCP, DEI and VID (VLAN id). Both
+-- PCP and DEI is always 0.  Inputs are in host byte-order, output is
+-- in network byte order.
+local function build_tag (vid, tpid)
+   return htonl(bit.bor(bit.lshift(tpid, 16), vid))
 end
 
 -- pop a VLAN tag (4 byte of TPID and TCI) from a packet
-function pop_tag(pkt)
+function pop_tag (pkt)
    local payload = pkt.data + o_ethernet_ethertype
    local length = pkt.length
    pkt.length = length - 4
    C.memmove(payload, payload + 4, length - o_ethernet_ethertype - 4)
 end
 
--- push a VLAN tag onto a packet
-function push_tag(pkt, tag)
+-- push a VLAN tag onto a packet.  The tag is in network byte-order.
+function push_tag (pkt, tag)
    local payload = pkt.data + o_ethernet_ethertype
    local length = pkt.length
    pkt.length = length + 4
@@ -52,15 +69,32 @@ function extract_tci(pkt)
 end
 
 -- extract VLAN id from TCI
-function tci_to_vid(tci)
+function tci_to_vid (tci)
    return bit.band(tci, 0xFFF)
 end
 
+function new_aux (self, conf)
+   local encap = conf.encapsulation
+   if (type(encap) == "string") then
+      self.tpid = tpids[encap]
+      assert(self.tpid, "Unsupported encapsulation type "..encap)
+   else
+      assert(type(encap) == "number")
+      self.tpid = encap
+   end
+   return self
+end
 
-function Tagger:new(conf)
+function check_tag (tag)
+   assert(tag > 0 and tag < 4095, "VLAN tag "..tag.." out of range")
+   return tag
+end
+
+function Tagger:new (conf)
    local o = setmetatable({}, {__index=Tagger})
-   o.tag = build_tag(assert(conf.tag))
-   return o
+   new_aux(o, conf)
+   o.tag = build_tag(check_tag(conf.tag), o.tpid)
+   return(o)
 end
 
 function Tagger:push ()
@@ -73,10 +107,11 @@ function Tagger:push ()
    end
 end
 
-function Untagger:new(conf)
+function Untagger:new (conf)
    local o = setmetatable({}, {__index=Untagger})
-   o.tag = build_tag(assert(conf.tag))
-   return o
+   new_aux(o, conf)
+   o.tag = build_tag(check_tag(conf.tag), o.tpid)
+   return(o)
 end
 
 function Untagger:push ()
@@ -95,45 +130,65 @@ function Untagger:push ()
    end
 end
 
+function VlanMux:new (conf)
+   local o = setmetatable({}, {__index=VlanMux})
+   return new_aux(o, conf)
+end
 
-VlanMux = {}
-function VlanMux:new()
-   local self = setmetatable({}, {__index=VlanMux})
-   self.dot1q_tpid = htons(dot1q_tpid)
-   return self
+function VlanMux:link ()
+   local from_vlans, to_vlans = {}, {}
+   for name, l in pairs(self.input) do
+      if string.match(name, "vlan%d+") then
+         local vid = check_tag(tonumber(string.sub(name, 5)))
+         to_vlans[vid] = self.output[name]
+         table.insert(from_vlans, { link = l, vid = vid })
+      elseif name == "native" then
+         to_vlans[0] = self.output.native
+      elseif type(name) == "string" and name ~= "trunk" then
+         error("invalid link name "..name)
+      end
+   end
+   self.from_vlans = from_vlans
+   self.to_vlans = to_vlans
 end
 
-function VlanMux:push()
-   local noutputs = #self.output
-   if noutputs > 0 then
-      for name, l in pairs(self.input) do
-         if type(name) == "string" then
-            for _ = 1, link.nreadable(l) do
-               local p = receive(l)
-               local ethertype = cast("uint16_t*", p.data + o_ethernet_ethertype)[0]
-
-               if name == "trunk" then -- trunk
-                  -- check for ethertype 0x8100 (802.1q VLAN tag)
-                  if ethertype == self.dot1q_tpid then
-                     -- dig out TCI field
-                     local tci = extract_tci(p)
-                     local vid = tci_to_vid(tci)
-                     local oif = self.output["vlan"..vid]
-                     pop_tag(p)
-                     self:transmit(oif, p)
-
-                  else -- untagged, send to native output
-                     self:transmit(self.output.native, p)
-                  end
-               elseif name == "native" then
-                  self:transmit(self.output.trunk, p)
-               else -- some vlanX interface
-                  local vid = tonumber(string.sub(name, 5))
-                  push_tag(p, build_tag(vid))
-                  self:transmit(self.output.trunk, p)
-               end
-            end
-         end
+function VlanMux:push ()
+   local from, to = self.from_vlans, self.to_vlans
+   local tpid = self.tpid
+   local l_in = self.input.trunk
+   assert(l_in)
+   while not empty(l_in) do
+      local p = receive(l_in)
+      local ethertype = cast("uint16_t*", p.data
+                                + o_ethernet_ethertype)[0]
+      if ethertype == htons(tpid) then
+         -- dig out TCI field
+         local tci = extract_tci(p)
+         local vid = tci_to_vid(tci)
+         pop_tag(p)
+         self:transmit(to[vid], p)
+      else -- untagged, send to native output
+         self:transmit(to[0], p)
+      end
+   end
+
+   local l_out = self.output.trunk
+   local i = 1
+   while from[i] do
+      local from = from[i]
+      local l_in = from.link
+      while not empty(l_in) do
+         local p = receive(l_in)
+         push_tag(p, build_tag(from.vid, tpid))
+         self:transmit(l_out, p)
+      end
+      i = i + 1
+   end
+
+   local l_in = self.input.native
+   if l_in then
+      while not empty(l_in) do
+         self:transmit(l_out, receive(l_in))
       end
    end
 end
@@ -161,7 +216,7 @@ function test_tag_untag ()
    local vid = 0
    for i=0,15 do
       for j=0,255 do
-         local tag = build_tag(vid)
+         local tag = build_tag(vid, tpids.dot1q)
          push_tag(pkt, tag)
          assert(cast(uint32_ptr_t, payload)[0] == tag)
          vid = vid + 1
@@ -176,17 +231,30 @@ function selftest()
    local basic_apps = require("apps.basic.basic_apps")
 
    local c = config.new()
-   config.app(c, "source", basic_apps.Source)
+   config.app(c, "vlan_source", basic_apps.Source)
    config.app(c, "vlan_mux", VlanMux)
-   config.app(c, "sink", basic_apps.Sink)
+   config.app(c, "trunk_sink", basic_apps.Sink)
+   config.app(c, "trunk_source", basic_apps.Source)
+   config.app(c, "native_source", basic_apps.Source)
+   config.app(c, "native_sink", basic_apps.Sink)
 
-   config.link(c, "source.output -> vlan_mux.vlan1")
-   config.link(c, "vlan_mux.trunk -> sink.input")
+   config.link(c, "vlan_source.output -> vlan_mux.vlan1")
+   config.link(c, "vlan_mux.trunk -> trunk_sink.input")
+   config.link(c, "trunk_source.output -> vlan_mux.trunk")
+   config.link(c, "vlan_mux.native -> native_sink.input")
+   config.link(c, "native_source.output -> vlan_mux.native")
    app.configure(c)
    app.main({duration = 1})
 
-   print("source sent: " .. link.stats(app.app_table.source.output.output).txpackets)
-   print("sink received: " .. link.stats(app.app_table.sink.input.input).rxpackets)
-
+   print("vlan sent: "
+            ..link.stats(app.app_table.vlan_source.output.output).txpackets)
+   print("native sent: "
+            ..link.stats(app.app_table.native_source.output.output).txpackets)
+   print("trunk received: "
+            ..link.stats(app.app_table.trunk_sink.input.input).rxpackets)
+   print("trunk sent: "
+            ..link.stats(app.app_table.trunk_source.output.output).txpackets)
+   print("native received: "
+            ..link.stats(app.app_table.native_sink.input.input).rxpackets)
    test_tag_untag()
 end