Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for IPv6 #410

Merged
merged 65 commits into from
Aug 14, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
65 commits
Select commit Hold shift + click to select a range
00d93ce
handled ipv6 nses in the ResolverConfig
phillip-stephens Jul 23, 2024
cba9372
added v6 local addrs to config
phillip-stephens Jul 23, 2024
02e09d7
added most basic IPv6 NS test
phillip-stephens Jul 29, 2024
6396707
added comments explaining we'll mix IPv4 and IPv6 addresses in the CL…
phillip-stephens Jul 29, 2024
fe07e25
CLI side done
phillip-stephens Jul 29, 2024
7eb62b7
init LocalAddr arrays in new RC and return error if ipv6only and we c…
phillip-stephens Jul 29, 2024
92c51fd
compiles with IPv6 support
phillip-stephens Jul 29, 2024
01c6d37
remove ipv6 todo
phillip-stephens Jul 29, 2024
8cc3bb4
fix bug in parsing IPv6 resolv.conf
phillip-stephens Jul 29, 2024
5494708
populate root servers with IPv6 if applicable
phillip-stephens Jul 29, 2024
b56781e
fixed population bug with ipv6 roots
phillip-stephens Jul 29, 2024
fb96b8f
fixed seg fault with a guard condition, need to fix root cause tho
phillip-stephens Jul 30, 2024
0529f12
added thread-saftey for PopulateAndValidate and fixed some lack of IP…
phillip-stephens Jul 30, 2024
a9981be
Merge branch 'main' into phillip/ipv6
phillip-stephens Jul 30, 2024
af3bd85
added a few sanity tests for IPv6
phillip-stephens Jul 30, 2024
c4ae1db
fixed bug by not deleting unneeded local addrs and nameservers
phillip-stephens Jul 30, 2024
23b0ae2
fixed executable name for testing
phillip-stephens Jul 30, 2024
5408aec
loopback handling in ExternalLoopback
phillip-stephens Jul 30, 2024
6274ac2
fixed compile issues in tests
phillip-stephens Jul 30, 2024
daa12f7
fixed bug with proper IPv6 detection and make populateAndValidate ide…
phillip-stephens Jul 31, 2024
a44af4f
fixed issue with copying ns arrays in resolver init
phillip-stephens Jul 31, 2024
b3c70da
added new --4 and --6 flags to disambiguate from the lookup A and AAA…
phillip-stephens Jul 31, 2024
78e1187
fix up unit tests, messed them up by misunderstanding the lookup-ipv4…
phillip-stephens Jul 31, 2024
2099ba7
fix up IPv6 test so it doesn't run on non-IPv6 supported hosts
phillip-stephens Jul 31, 2024
82a9834
forgot to rename the ZDNS exe back
phillip-stephens Jul 31, 2024
64e58c6
moved ipv6 tests into their own workflow
phillip-stephens Jul 31, 2024
364b34e
renamed zdns exe
phillip-stephens Jul 31, 2024
b956318
cleaned up ipv6 tests
phillip-stephens Jul 31, 2024
977c18e
Merge branch 'main' into phillip/ipv6
phillip-stephens Jul 31, 2024
9df8ad8
added loopback test for ipv6
phillip-stephens Jul 31, 2024
6ea918e
elevate warning msg about not being able to find IPv6 socket to warni…
phillip-stephens Jul 31, 2024
844890f
add fix for using root servers if in iterative mode in CLI
phillip-stephens Aug 1, 2024
05e9a8e
added details to loopback warning msg
phillip-stephens Aug 1, 2024
c60734b
lookup AAAA for nameservers in extract authority
phillip-stephens Aug 1, 2024
3b5794d
added prefer ipv4 and ipv6 options
phillip-stephens Aug 1, 2024
9499bfb
better handling around the ipv4 preference validation, better UX, mor…
phillip-stephens Aug 2, 2024
5208e8a
Merge branch 'main' into phillip/ipv6
phillip-stephens Aug 2, 2024
77afc83
lint
phillip-stephens Aug 2, 2024
3dabf8d
hack to get IPv6 test to pass on hosts that don't support IPv6
phillip-stephens Aug 2, 2024
e3b4166
a lil more hack
phillip-stephens Aug 2, 2024
644b544
added comment to explain hack in unit tests
phillip-stephens Aug 2, 2024
b7f67c0
Merge branch 'main' into phillip/ipv6
phillip-stephens Aug 5, 2024
8dd4b64
Merge branch 'main' into phillip/ipv6
phillip-stephens Aug 7, 2024
6cb473d
fixed compile issues in non-test code
phillip-stephens Aug 7, 2024
28377db
tests passing
phillip-stephens Aug 7, 2024
a082f14
lint
phillip-stephens Aug 7, 2024
a843f6a
remove tests with loopback IPv6, we can't have IPv6 loopback NSs
phillip-stephens Aug 7, 2024
5540069
only add name servers if IP mode supports it
phillip-stephens Aug 7, 2024
2f10f2d
cleanup and fixed some bugs from merge
phillip-stephens Aug 7, 2024
f0c53d1
loopback cli hack and added more string sanitization on listing --nam…
phillip-stephens Aug 7, 2024
a65c975
don't overwrite cli provided external NS'
phillip-stephens Aug 7, 2024
a805bfa
don't overwrite cli provided external NS bug
phillip-stephens Aug 7, 2024
51e078f
Merge branch 'main' into phillip/ipv6
phillip-stephens Aug 9, 2024
d38e5fe
compiling
phillip-stephens Aug 9, 2024
50fef99
fixed up tests and added null checks to ExternalLookup. Removed test …
phillip-stephens Aug 12, 2024
536e9c2
cleaned up unneeded changes in integration_tests
phillip-stephens Aug 12, 2024
970aa63
spelling
phillip-stephens Aug 12, 2024
f11791f
use new concat
phillip-stephens Aug 12, 2024
77784db
avoid redundent check and remove todo
phillip-stephens Aug 12, 2024
c4ca392
Stop handling a domain if all nameservers don't provide sufficient gl…
phillip-stephens Aug 13, 2024
e9b675f
better err msg if user specifies IPv6 mode on non IPv6 capable machine
phillip-stephens Aug 13, 2024
3a72edd
infer IP support thru nameservers, use --4/6 as IPvX only
phillip-stephens Aug 14, 2024
433bd3c
disallow both --4 and --6
phillip-stephens Aug 14, 2024
a71f07a
tests and lints
phillip-stephens Aug 14, 2024
1496962
fixed up ipv6 tests
phillip-stephens Aug 14, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions makefile
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,11 @@ integration-tests: zdns
python3 testing/integration_tests.py
python3 testing/large_scan_integration/large_scan_integration_tests.py

# Not all hosts support this, so this will be a custom make target
ipv6-tests: zdns
pip3 install -r testing/requirements.txt
python3 testing/ipv6_tests.py

lint:
goimports -w -local "github.com/zmap/zdns" ./
gofmt -s -w ./
Expand Down
5 changes: 3 additions & 2 deletions src/cli/alookup.go
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@
* implied. See the License for the specific language governing
* permissions and limitations under the License.
*/

package cli

import (
Expand Down Expand Up @@ -40,8 +41,8 @@ Specifically, alookup acts similar to nslookup and will follow CNAME records.`,
func init() {
rootCmd.AddCommand(alookupCmd)

alookupCmd.PersistentFlags().Bool("ipv4-lookup", false, "perform A lookups for each MX server")
alookupCmd.PersistentFlags().Bool("ipv6-lookup", false, "perform AAAA record lookups for each MX server")
alookupCmd.PersistentFlags().Bool("ipv4-lookup", false, "perform A lookups for each server")
alookupCmd.PersistentFlags().Bool("ipv6-lookup", false, "perform AAAA record lookups for each server")

util.BindFlags(alookupCmd, viper.GetViper(), util.EnvPrefix)
}
13 changes: 11 additions & 2 deletions src/cli/cli.go
Original file line number Diff line number Diff line change
Expand Up @@ -69,6 +69,10 @@ type CLIConf struct {
LookupAllNameServers bool
TCPOnly bool
UDPOnly bool
IPv4TransportOnly bool // IPv4 transport only, incompatible with IPv6 transport only
IPv6TransportOnly bool // IPv6 transport only, incompatible with IPv4 transport only
PreferIPv4Iteration bool // Prefer IPv4/A record lookups during iterative resolution, only used if both IPv4 and IPv6 transport are enabled
PreferIPv6Iteration bool // Prefer IPv6/AAAA record lookups during iterative resolution, only used if both IPv4 and IPv6 transport are enabled
RecycleSockets bool
LocalAddrSpecified bool
LocalAddrs []net.IP
Expand Down Expand Up @@ -165,6 +169,11 @@ func init() {
rootCmd.PersistentFlags().StringVar(&GC.NameServersString, "name-servers", "", "List of DNS servers to use. Can be passed as comma-delimited string or via @/path/to/file. If no port is specified, defaults to 53.")
rootCmd.PersistentFlags().StringVar(&GC.LocalAddrString, "local-addr", "", "comma-delimited list of local addresses to use, serve as the source IP for outbound queries")
rootCmd.PersistentFlags().StringVar(&GC.LocalIfaceString, "local-interface", "", "local interface to use")
rootCmd.PersistentFlags().BoolVar(&GC.IPv4TransportOnly, "4", false, "utilize IPv4 query transport only, incompatible with --6")
rootCmd.PersistentFlags().BoolVar(&GC.IPv6TransportOnly, "6", false, "utilize IPv6 query transport only, incompatible with --4")
rootCmd.PersistentFlags().BoolVar(&GC.PreferIPv4Iteration, "prefer-ipv4-iteration", false, "Prefer IPv4/A record lookups during iterative resolution. Ignored unless used with both IPv4 and IPv6")
rootCmd.PersistentFlags().BoolVar(&GC.PreferIPv6Iteration, "prefer-ipv6-iteration", false, "Prefer IPv6/AAAA record lookups during iterative resolution. Ignored unless used with both IPv4 and IPv6")

rootCmd.PersistentFlags().StringVar(&GC.ConfigFilePath, "conf-file", zdns.DefaultNameServerConfigFile, "config file for DNS servers")
rootCmd.PersistentFlags().IntVar(&GC.Timeout, "timeout", 15, "timeout for resolving a individual name, in seconds")
rootCmd.PersistentFlags().IntVar(&GC.IterationTimeout, "iteration-timeout", 4, "timeout for a single iterative step in an iterative query, in seconds. Only applicable with --iterative")
Expand All @@ -174,8 +183,8 @@ func init() {
rootCmd.PersistentFlags().BoolVar(&GC.Dnssec, "dnssec", false, "Requests DNSSEC records by setting the DNSSEC OK (DO) bit")
rootCmd.PersistentFlags().BoolVar(&GC.UseNSID, "nsid", false, "Request NSID.")

rootCmd.PersistentFlags().Bool("ipv4-lookup", false, "Perform an IPv4 Lookup in modules")
rootCmd.PersistentFlags().Bool("ipv6-lookup", false, "Perform an IPv6 Lookup in modules")
rootCmd.PersistentFlags().Bool("ipv4-lookup", false, "Perform an IPv4 Lookup (requests A records) in modules")
rootCmd.PersistentFlags().Bool("ipv6-lookup", false, "Perform an IPv6 Lookup (requests AAAA recoreds) in modules")
rootCmd.PersistentFlags().StringVar(&GC.BlacklistFilePath, "blacklist-file", "", "blacklist file for servers to exclude from lookups")
}

Expand Down
15 changes: 11 additions & 4 deletions src/cli/config_validation.go
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,7 @@ func populateNetworkingConfig(gc *CLIConf) error {
return errors.Wrap(err, "client subnet did not pass validation")
}

// local address - the user can enter both IPv4 and IPv6 addresses. We'll differentiate them later
if GC.LocalAddrString != "" {
for _, la := range strings.Split(GC.LocalAddrString, ",") {
ip := net.ParseIP(la)
Expand All @@ -51,6 +52,7 @@ func populateNetworkingConfig(gc *CLIConf) error {
gc.LocalAddrSpecified = true
}

// local interface - same as local addresses, an interface could have both IPv4 and IPv6 addresses, we'll differentiate them later
if gc.LocalIfaceString != "" {
li, err := net.InterfaceByName(gc.LocalIfaceString)
if err != nil {
Expand Down Expand Up @@ -114,7 +116,7 @@ func parseNameServers(gc *CLIConf) error {
if gc.NameServerMode {
log.Fatal("name servers cannot be specified on command line in --name-server-mode")
}
var ns []string
var nses []string
if (gc.NameServersString)[0] == '@' {
filepath := (gc.NameServersString)[1:]
f, err := os.ReadFile(filepath)
Expand All @@ -124,11 +126,16 @@ func parseNameServers(gc *CLIConf) error {
if len(f) == 0 {
log.Fatalf("Empty file (%s)", filepath)
}
ns = strings.Split(strings.Trim(string(f), "\n"), "\n")
nses = strings.Split(strings.Trim(string(f), "\n"), "\n")
} else {
ns = strings.Split(gc.NameServersString, ",")
nses = strings.Split(gc.NameServersString, ",")
trimmedNSes := make([]string, 0, len(nses))
for _, ns := range nses {
trimmedNSes = append(trimmedNSes, strings.TrimSpace(ns))
}
nses = trimmedNSes
}
gc.NameServers = ns
gc.NameServers = nses
}
return nil
}
9 changes: 6 additions & 3 deletions src/cli/config_validation_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -22,22 +22,25 @@ import (
func TestValidateNetworkingConfig(t *testing.T) {
t.Run("LocalAddr and LocalInterface both specified", func(t *testing.T) {
gc := &CLIConf{
LocalAddrString: "1.1.1.1",
LocalIfaceString: "eth0",
LocalAddrString: "1.1.1.1",
LocalIfaceString: "eth0",
IPv4TransportOnly: true,
}
err := populateNetworkingConfig(gc)
require.NotNil(t, err, "Expected an error but got nil")
})
t.Run("Using invalid interface", func(t *testing.T) {
gc := &CLIConf{
LocalIfaceString: "invalid_interface",
LocalIfaceString: "invalid_interface",
IPv4TransportOnly: true,
}
err := populateNetworkingConfig(gc)
require.NotNil(t, err, "Expected an error but got nil")
})
t.Run("Using nameserver with port", func(t *testing.T) {
gc := &CLIConf{
NameServersString: "127.0.0.1:53",
IPv4TransportOnly: true,
}
err := populateNetworkingConfig(gc)
require.Nil(t, err, "Expected no error but got %v", err)
Expand Down
Loading