converter.md

iptables to JSON Converter

While designing opa-iptables extension, we knew that we have to store the iptables rules to OPA for persistence and also querying these rules using policy. We could have simply stored raw rules in JSON as following:

{
    rules : [
        "iptables -t FILTER -A INPUT -p tcp --dport 9090 -j DROP",
        "iptables -t FILTER -A INPUT -p tcp --dport 33455 -j ACCEPT"
    ]
}

But, at that time multiple issues were raised about this approach:

1. The underlying library that we are using for installing these rules to the kernel is coreos's go-iptables. And it's API for inserting/deleting rules do not take whole iptables rule but it requires []string as an input.

2. We want to do some lexical analysis of these rules before inserting it to the kernel.

3. Currently, OPA only supports JSON and YAML format for storing data.

Hence, We decided to go with a more structured way of representing these rules. Now you know the story why we decided to go with this particular approach. The following document describes how it all works.

Implementaion

We have a Go struct called Rule which describes a single iptables rule.

type Rule struct {
    Table              string   `json:"table"`
    Chain              string 
    DestinationPort    string 
    DestinationAddress string 
    DestinationRange   string 
    SourceAddress      string 
    SourcePort         string 
    SourceRange        string
    ...
}

This is the most important struct in the sense that every iptables rule related functionality like inserting/deleting or marshaling/unmarshaling rule in Go is wrapped around this struct.

So, how we are going

From:

iptables -t FILTER -A INPUT -p tcp --dport 9090 -j DROP -m comment --comment "drop all traffic to web server

To:

{
    "table": "filter",
    "chain": "INPUT",
    "destination_port": "9090",
    "jump": "DROP",
    "protocol": "tcp",
    "tcp_flags": {},
    "ctstate": [
        ""
    ],
    "match": [
        "comment"
    ],
    "comment": "drop all traffic to web servern"
}

Now you know that we have to somehow need to create an instance of Rule struct. Then, Using Go's encoding/json package we can marshal this struct to JSON using json.Marshal() function.

For parsing of rule, I have decided to use Go's flag package. While using it I have realized that it's not going to work because of the following issues:

1. Some of the arguments in the rule may have more than one number of values. i.e.

   iptables --tcp-flags  ACK  FIN,RST,SYN <---- 2 values
                       |---| |----------|
   
   iptables --dport 8080 <---- 1 value
   

2. Go's flag package works with OS Args.

For these reasons, I decided to create a custom flag pkg based on Go's flag pkg which satisfy all of our requirements.

Major changes are:

1. Each flag can describe how many args it has.

   type Flag struct {
       name    string
       value   value  
       numArgs int   <--- number of arguments a flag requires
   }
   

2. During parsing that flag it respects this numArgs constraint.

       actualValue := ""
       numArg := flag.numArgs
       if len(fs.args)-flag.numArgs >= 0 {
           value, fs.args = fs.args[:numArg], fs.args[numArg:]
           for _, v := range value {
               if len(v) > 0 && v[0] == '-' {
                   return false, argumentError{name, numArg}
               }
           }
           hasArgs = true
           actualValue = strings.Join(value, "#")
       }
   

The way parser work is, it uses flagset which contains all the flags we are wanted to parse. Following is a struct which describes all flags:

// IPTableflagSet represents all possible iptables flag that we support currently.
type IPTableflagSet struct {
    TableFlag        string
    ChainFlag        string
    ProtocolFlag     string
    SourceFlag       string
    DestinationFlag  string
    DportFlag        string
    SportFlag        string
    InInterfaceFlag  string
    OutInterfaceFlag string
    DesRangeFlag     string
    SrcRangeFlag     string
    JumpFlag         string
    MatchFlag        string
    ToPortFlag       string
    CTStateFlag      string
    Comment          string

    TCPFlag TCPFlags
}

IPTableflagSet is initialized in the following function:

// InitFlagSet Adds user defined Flag into FlagSet.
func (fs *FlagSet) InitFlagSet(tf *IPTableflagSet) {
    fs.AddStringFlag(&tf.TableFlag, "t", "", 1)
    fs.AddStringFlag(&tf.ChainFlag, "A", "", 1)
    fs.AddStringFlag(&tf.ChainFlag, "I", "", 1)
    fs.AddFlag(&tf.TCPFlag, "tcp-flags", 2)
    fs.AddStringFlag(&tf.CTStateFlag, "ctstate", "", 1)
    fs.AddStringFlag(&tf.Comment, "comment", "", 1)
    ...
}

How to Add new iptable flag to flagset for parsing?

Let's suppose we are wanted to add an iptable flag called --log-prefix. It has only one argument and its type is string.

1. Add a new field to Rule struct

type Rule struct {
    ...

+   LogPrefix  string  `json:"log-prefix,omitempty"`

}

2. Now add this flag to our flagset, so the parser can know it and able to parse it.

type IPTableflagSet struct {
    ...

+   LogPrefixFlag  string

}

func (fs *FlagSet) InitFlagSet(tf *IPTableflagSet) {
    ...

+   fs.AddStringFlag(&tf.LogPrefixFlag, "log-prefix", "", 1)

}

Done 👍. You have successfully added a new iptable flag to our existing iptable flagSet 🎉.