x/net/idna: separate CheckHyphens from ValidateLabel

[TimothyGu]

What version of Go are you using (go version)?

$ go version
go version go1.15.2 linux/amd64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

go env Output

$ go env
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/timothy-gu/.cache/go-build"
GOENV="/home/timothy-gu/.config/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOINSECURE=""
GOMODCACHE="/home/timothy-gu/go/pkg/mod"
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPATH="/home/timothy-gu/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/usr/lib/go"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/lib/go/pkg/tool/linux_amd64"
GCCGO="gccgo"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/home/timothy-gu/dev/go/url/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build009079035=/tmp/go-build -gno-record-gcc-switches"

What did you do?

Since UTS #46 version 10.0.0, ToASCII has had the CheckHyphens option to control checking for hyphens in the validation stage of IDNA conversion independently from other options. Indeed, the URL standard for web browsers simultaneously has these two requirements:

1. error out on "https://al\u200Cp.com/"
2. but allow "http://r2---sn-gvbxgn-tt1s.googlevideo.com/"

This behavior is seen in Node.js as well as WebKit-based browsers like Safari. Here's the output from Node.js:

> new URL("https://al\u200Cp.com/")
Uncaught TypeError [ERR_INVALID_URL]: Invalid URL: https://al‌p.com/
> new URL("http://r2---sn-gvbxgn-tt1s.googlevideo.com/")
URL { ... }

However, x/net/idna does not allow controlling these two cases independently. When the ValidateLabels option is specified, the package would treat both as invalid; and if it's not specified, the package would treat both as valid: https://play.golang.org/p/O-nRi96pQH3

What did you expect to see?

x/net/idna should allow disabling hyphen checking independently of other label validation routines.

What did you see instead?

x/net/idna couples hyphen checking with other label validation routines.

[gopherbot]

Change https://golang.org/cl/258837 mentions this issue: internal/export/idna: Allow specifying CheckHyphens and CheckJoiners

[andybons]

@nigeltao @mpvl @bradfitz

[janikrabe]

Thank you @TimothyGu. At Cloudflare some of our customers are seeing issues because of this that your change would allow us to fix. Let us know if we can be of any help.

[TimothyGu]

I've got a CL open: https://golang.org/cl/258837. It just needs to be reviewed. Ping @mpvl again.

[TimothyGu]

Okay, the x/text CL is in. We still need a x/net CL to sync the generated files with x/text.

[gopherbot]

Change https://golang.org/cl/273586 mentions this issue: idna: allow specifying CheckHyphens and CheckJoiners

[janik-cloudflare]

@mpvl and @nigeltao, any chance you could take a look at the x/net CL as well? 🙏 It would be great if we could get this merged into the February 2022 release.

[janik-cloudflare]

Thank you all!