vhost doesn't work correctly in the http protocol in net/http package

[s4hm4d]

What version of Go are you using (go version)?

$ go version
go version go1.20.1 linux/amd64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

go env Output

$ go env
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/myuser/.cache/go-build"
GOENV="/home/myuser/.config/go/env"
GOEXE=""
GOEXPERIMENT=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOINSECURE=""
GOMODCACHE="/home/myuser/go/pkg/mod"
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPATH="/home/myuser/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/usr/local/go"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/local/go/pkg/tool/linux_amd64"
GOVCS=""
GOVERSION="go1.20.1"
GCCGO="gccgo"
GOAMD64="v1"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/dev/null"
GOWORK=""
CGO_CFLAGS="-O2 -g"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-O2 -g"
CGO_FFLAGS="-O2 -g"
CGO_LDFLAGS="-O2 -g"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -Wl,--no-gc-sections -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build2228368994=/tmp/go-build -gno-record-gcc-switches"

What did you do?

I wrote a simple script to test virtual host.
And added a proxy address to check the requests. I used Burp Suite. Also tested with Postman and other addresses. no difference.

It doesn't work correctly for the HTTP protocol when using the proxy.
If use the HTTPS protocol or comment the proxy address in the HTTP protocol, It works correctly.

For example:

$ ping wikipedia.org
PING wikipedia.org (91.198.174.192) 56(84) bytes of data.

$ ping github.com
PING github.com (140.82.121.3) 56(84) bytes of data.

If I send a request to the Wikipedia server with the Host: github.com, I will receive an error.

$ curl http://91.198.174.192 -H "Host: github.com" -I
HTTP/1.1 400 
Date: Mon, 20 Feb 2023 04:23:45 GMT
Server: Varnish
X-Varnish: 1017982837
X-Cache: cp3062 int
X-Cache-Status: int-front
Server-Timing: cache;desc="int-front", host;desc="cp3062"
Permissions-Policy: interest-cohort=()
Set-Cookie: WMF-Last-Access=20-Feb-2023;Path=/;HttpOnly;secure;Expires=Fri, 24 Mar 2023 00:00:00 GMT
Set-Cookie: WMF-Last-Access-Global=20-Feb-2023;Path=/;Domain=.invalid;HttpOnly;secure;Expires=Fri, 24 Mar 2023 00:00:00 GMT
X-Client-IP: 217.66.208.107
Content-Type: text/html; charset=utf-8
Content-Length: 1806
Connection: keep-alive

It's 400 status code and It's an error.

Also I sent the request to the proxy (Burp Suite):

This is a Golang script to check this:

vhostTest.go Script

package main
import (

"fmt"

"net/http"

"crypto/tls"

"net/url"

)
func main() {

proxyStr := "http://127.0.0.1:8080"

proxyURL, _ := url.Parse(proxyStr)
requestURL := "http://91.198.174.192"

req, err := http.NewRequest("GET", requestURL, nil)
if err != nil {
	fmt.Println(err)
	return
}

req.Host = "github.com"

transport := &http.Transport{
	Proxy: http.ProxyURL(proxyURL),
	TLSClientConfig: &tls.Config{InsecureSkipVerify: true},
}

client := &http.Client{
	Transport: transport,
	CheckRedirect: func(req *http.Request, via []*http.Request) error {
		return http.ErrUseLastResponse
	},
}

resp, err := client.Do(req)
if err != nil {
	fmt.Println(err)
	return
}

fmt.Println(resp.StatusCode)
for name, values := range resp.Header {
	for _, value := range values {
		fmt.Println(name, value)
	}
}

}

First, I commented the proxy address and run the script:

$ go run vhostTest.go 
400
Server Varnish
X-Cache-Status int-front
Server-Timing cache;desc="int-front", host;desc="cp3062"
Content-Type text/html; charset=utf-8
Content-Length 1806
Date Mon, 20 Feb 2023 04:32:52 GMT
X-Cache cp3062 int
Permissions-Policy interest-cohort=()
Set-Cookie WMF-Last-Access=20-Feb-2023;Path=/;HttpOnly;secure;Expires=Fri, 24 Mar 2023 00:00:00 GMT
Set-Cookie WMF-Last-Access-Global=20-Feb-2023;Path=/;Domain=.invalid;HttpOnly;secure;Expires=Fri, 24 Mar 2023 00:00:00 GMT
Connection keep-alive
X-Varnish 1025237235
X-Client-Ip 217.66.208.107

The output is correct.

Next run is with the proxy address:

$ go run vhostTest.go 
301
Content-Length 0
Location https://github.com/

This is wrong, because the request sent to http://github.com instead of http://91.198.174.192

Next run is with the proxy address and the HTTPS protocol:

$ go run vhostTest.go 
400
Content-Length 1804
X-Cache cp3062 int
Content-Type text/html; charset=utf-8
Server Varnish
X-Cache-Status int-front
Permissions-Policy interest-cohort=()
Date Mon, 20 Feb 2023 06:19:36 GMT
X-Client-Ip 217.66.208.107
Set-Cookie WMF-Last-Access=20-Feb-2023;Path=/;HttpOnly;secure;Expires=Fri, 24 Mar 2023 00:00:00 GMT
Set-Cookie WMF-Last-Access-Global=20-Feb-2023;Path=/;Domain=.invalid;HttpOnly;secure;Expires=Fri, 24 Mar 2023 00:00:00 GMT
Server-Timing cache;desc="int-front", host;desc="cp3062"

The output is correct.

What did you expect to see?

When using the proxy address in the HTTP protocol, I expect the 400 status code, like the curl.

What did you see instead?

It ignore the server IP and the request will be sent to the host value address.

[jimen0]

Hi @s4hm4d - this seems to be working as documented. Per https://cs.opensource.google/go/go/+/refs/tags/go1.20.1:src/net/http/request.go;l=238-242:

	// For client requests, Host optionally overrides the Host
	// header to send. If empty, the Request.Write method uses
	// the value of URL.Host. Host may contain an international
	// domain name.

If you want Go's http client to send the request to the IP instead, just follow the docs and don't set the http.Request.Host field (req.Host = "github.com" on your code). I checked this with Burp and can confirm the request you expected is issued.

[s4hm4d]

Hi @jimen0
This isn't working as documented.
Because it has different output based on the protocol (HTTP or HTTPS) and the proxy server.

Due to this issue many tools that written with Golang don't work correctly.
For example, I want to fuzz virtual hosts with ffuf:
I create domains.txt with this content:

google.com
wikipedia.org
github.com
stackoverflow.com

And run the ffuf:

$ ffuf -u http://91.198.174.192 -H "Host: FUZZ" -w domains.txt

Output:

wikipedia.org           [Status: 301, Size: 0, Words: 1, Lines: 1, Duration: 121ms]

The output is correct, because I used the Wikipedia server IP.
Note that the protocol is HTTP

Now, I run it again, just add the burp suite address as the proxy:

$ ffuf -u http://91.198.174.192 -H "Host: FUZZ" -w domains.txt -x http://127.0.0.1:8080

Output:

github.com              [Status: 301, Size: 0, Words: 1, Lines: 1, Duration: 263ms]
wikipedia.org           [Status: 301, Size: 0, Words: 1, Lines: 1, Duration: 376ms]
stackoverflow.com       [Status: 301, Size: 0, Words: 1, Lines: 1, Duration: 413ms]
google.com              [Status: 301, Size: 219, Words: 9, Lines: 7, Duration: 546ms]

Note again that the protocol is HTTP
The output is incorrect! I didn't change anything. just add a proxy. but the result is incorrect. because when using the proxy in HTTP requests, it ignore the URLs. If it's working as documented, we should get the same result with the proxy and without it.

I run it again in HTTPS protocol and using the proxy:

$ ffuf -u https://91.198.174.192 -H "Host: FUZZ" -w domains.txt -x http://127.0.0.1:8080

Output:

wikipedia.org           [Status: 301, Size: 234, Words: 14, Lines: 8, Duration: 1451ms]

The output is correct!
Note that the protocol is HTTPS

[thanm]

@neild per owners.

Just as a general remark, please use plain text when including examples/transcripts in your issue, as opposed to pictures/screendumps, a lot of what you have posted is difficult to read.

[neild]

You're getting different results with HTTP and HTTPS requests through a proxy because HTTP requests are sent unencrypted to the proxy while HTTPS requests request a tunnel to the destination with CONNECT.

• http://10.0.0.1/, Host = foo.com, no proxy: dial 10.0.0.1, request http://foo.com/.
• https://10.0.0.1/, Host = foo.com, no proxy: dial 10.0.0.1 with TLS, request https://foo.com/.
• http://10.0.0.1/, Host = foo.com, proxy: send request for http://foo.com/ to the proxy. Note that there is no way to request that a proxy send a request to a specific IP address with a specific Host header; see https://www.rfc-editor.org/rfc/rfc9112#section-3.2.2-7.
• https://10.0.0.1/, Host = foo.com, proxy: request a proxy tunnel to 10.0.0.1 with CONNECT, use TLS on that tunnel, send request for https://foo.com/.

[seankhliao]

looks to be working as intended