Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

HandleDir AssestValidator: http/2.0 push seems not working #1562

Closed
xiaozhuai opened this issue Jul 15, 2020 · 13 comments
Closed

HandleDir AssestValidator: http/2.0 push seems not working #1562

xiaozhuai opened this issue Jul 15, 2020 · 13 comments

Comments

@xiaozhuai
Copy link

xiaozhuai commented Jul 15, 2020

Describe the bug

Here is my code snippets

	assetNames := GzipAssetNames()
	pushTargets := make([]string, 0)
	for _, name := range assetNames {
		// ignore *.map file
		if !strings.HasSuffix(name, ".map") {
			pos := strings.Index(name, "public/")
			target := name[pos+6:]
			// skip /index.html
			if target != "/index.html" {
				pushTargets = append(pushTargets, target)
			}
		}
	}

	app.HandleDir("/", "public", iris.DirOptions{
		Asset:      GzipAsset,
		AssetInfo:  GzipAssetInfo,
		AssetNames: GzipAssetNames,
		AssetValidator: func(ctx iris.Context, name string) bool {
			ctx.Header("Vary", "Accept-Encoding")
			ctx.Header("Content-Encoding", "gzip")

			if strings.HasSuffix(name, "public/index.html") {
				for _, target := range pushTargets {
					app.Logger().Infof("!!!! Push %s", target)
					err := ctx.ResponseWriter().Push(target, nil)
					if err != nil {
						break
					}
				}
			}

			return true
		},
	})

The log shows

[INFO] 2020/07/15 23:49 !!!! Push /fonts/element-icons.535877f5.woff
[INFO] 2020/07/15 23:49 !!!! Push /js/app.3efd4a6d.js
[INFO] 2020/07/15 23:49 !!!! Push /css/app.e275010e.css
[INFO] 2020/07/15 23:49 !!!! Push /favicon.ico
[INFO] 2020/07/15 23:49 200 44.663µs 127.0.0.1 GET /css/app.e275010e.css
[INFO] 2020/07/15 23:49 200 24.339µs 127.0.0.1 GET /favicon.ico
[INFO] 2020/07/15 23:49 !!!! Push /fonts/element-icons.732389de.ttf
[INFO] 2020/07/15 23:49 200 472.742µs 127.0.0.1 GET /fonts/element-icons.535877f5.woff
[INFO] 2020/07/15 23:49 !!!! Push /img/nmac.e2c19df3.png
[INFO] 2020/07/15 23:49 200 495.606µs 127.0.0.1 GET /js/app.3efd4a6d.js
[INFO] 2020/07/15 23:49 !!!! Push /js/chunk-vendors.3a7bf853.js
[INFO] 2020/07/15 23:49 !!!! Push /css/chunk-vendors.e82fddda.css
[INFO] 2020/07/15 23:49 200 1.381941ms 127.0.0.1 GET /fonts/element-icons.732389de.ttf
[INFO] 2020/07/15 23:49 200 2.308779ms 127.0.0.1 GET /
[INFO] 2020/07/15 23:49 200 1.417537ms 127.0.0.1 GET /img/nmac.e2c19df3.png
[INFO] 2020/07/15 23:49 200 242.378µs 127.0.0.1 GET /css/chunk-vendors.e82fddda.css
[INFO] 2020/07/15 23:49 200 999.125µs 127.0.0.1 GET /js/chunk-vendors.3a7bf853.js
[INFO] 2020/07/15 23:49 200 64.567µs 127.0.0.1 GET /css/app.e275010e.css
[INFO] 2020/07/15 23:49 200 187.796µs 127.0.0.1 GET /css/chunk-vendors.e82fddda.css
[INFO] 2020/07/15 23:49 200 154.553µs 127.0.0.1 GET /js/app.3efd4a6d.js
[INFO] 2020/07/15 23:49 200 333.652µs 127.0.0.1 GET /js/chunk-vendors.3a7bf853.js
[INFO] 2020/07/15 23:49 200 160.891µs 127.0.0.1 GET /img/nmac.e2c19df3.png
[INFO] 2020/07/15 23:49 200 267.272µs 127.0.0.1 GET /fonts/element-icons.535877f5.woff
[INFO] 2020/07/15 23:49 200 167.745µs 127.0.0.1 GET /api/categories
[INFO] 2020/07/15 23:49 200 225.605µs 127.0.0.1 GET /js/app.3efd4a6d.js.map
[INFO] 2020/07/15 23:49 200 1.052831ms 127.0.0.1 GET /js/chunk-vendors.3a7bf853.js.map
[INFO] 2020/07/15 23:49 200 45.121µs 127.0.0.1 GET /favicon.ico

But the Chrome Initiator do not hint Push

image

@kataras
Copy link
Owner

kataras commented Jul 15, 2020

@xiaozhuai If you are going to write on AssetValidator then you SHOULD return false not true, because returning true will make the app.HandleDir serve the file by its own.

@kataras kataras removed their assignment Jul 15, 2020
@kataras kataras changed the title [BUG] http/2.0 push seems not working HandleDir AssestValidator: http/2.0 push seems not working Jul 15, 2020
@kataras
Copy link
Owner

kataras commented Jul 15, 2020

Also, that could be feature request too, we can add an option to handle Push on index file's assets or defined slice of filenames.

@xiaozhuai
Copy link
Author

xiaozhuai commented Jul 16, 2020

	assetNames := GzipAssetNames()
	pushTargets := make([]string, 0)
	for _, name := range assetNames {
		// ignore *.map file
		if !strings.HasSuffix(name, ".map") {
			pos := strings.Index(name, "public/")
			target := name[pos+6:]
			// skip /index.html
			if target != "/index.html" {
				pushTargets = append(pushTargets, target)
			}
		}
	}

	app.Use(func(ctx iris.Context) {
		if ctx.Request().RequestURI == "/" {
			for _, target := range pushTargets {
				app.Logger().Infof("!!!! Push %s", target)
				err := ctx.ResponseWriter().Push(target, nil)
				if err != nil {
					break
				}
			}
		}
		ctx.Next()
	})

	app.HandleDir("/", "public", iris.DirOptions{
		Asset:      GzipAsset,
		AssetInfo:  GzipAssetInfo,
		AssetNames: GzipAssetNames,
		AssetValidator: func(ctx iris.Context, name string) bool {
			ctx.Header("Vary", "Accept-Encoding")
			ctx.Header("Content-Encoding", "gzip")
			return true
		},
	})
	assetNames := GzipAssetNames()
	pushTargets := make([]string, 0)
	for _, name := range assetNames {
		// ignore *.map file
		if !strings.HasSuffix(name, ".map") {
			pos := strings.Index(name, "public/")
			target := name[pos+6:]
			// skip /index.html
			if target != "/index.html" {
				pushTargets = append(pushTargets, target)
			}
		}
	}

	app.HandleDir("/", "public", iris.DirOptions{
		Asset:      GzipAsset,
		AssetInfo:  GzipAssetInfo,
		AssetNames: GzipAssetNames,
		AssetValidator: func(ctx iris.Context, name string) bool {
			ctx.Header("Vary", "Accept-Encoding")
			ctx.Header("Content-Encoding", "gzip")

			if strings.HasSuffix(name, "public/index.html") {
				for _, target := range pushTargets {
					app.Logger().Infof("!!!! Push %s", target)
					err := ctx.ResponseWriter().Push(target, nil)
					if err != nil {
						break
					}
				}

				data, _ := GzipAsset(name)
				ctx.Write(data)
				return false
			}

			return true
		},
	})

Thanks @kataras ! And I came up with these two, but both not work. Why?

@kataras
Copy link
Owner

kataras commented Jul 16, 2020

Give me some time @xiaozhuai, I will push an example with a new feature for that one ^

EDIT: you are running the Iris server under TLS right? (Push feature requires http/2-tls)

@xiaozhuai
Copy link
Author

@kataras Yes,I run iris with TLS

@kataras
Copy link
Owner

kataras commented Jul 16, 2020

@xiaozhuai Here you are:

var opts = iris.DirOptions{
IndexName: "/index.html",
PushTargets: map[string][]string{
"/": {
"/public/favicon.ico",
"/public/js/main.js",
"/public/css/main.css",
},
},
Compress: false, // SHOULD be set to false, files already compressed.
ShowList: true,
Asset: GzipAsset,
AssetInfo: GzipAssetInfo,
AssetNames: GzipAssetNames,
// Required for pre-compressed files:
AssetValidator: func(ctx iris.Context, _ string) bool {
ctx.Header("Vary", "Content-Encoding")
ctx.Header("Content-Encoding", "gzip")
return true
},
}
func main() {
app := iris.New()
app.HandleDir("/public", "./assets", opts)
app.Run(iris.TLS(":443", "../http2push/mycert.crt", "../http2push/mykey.key"))
}

  • Upgrade Iris to its master go get -u github.com/kataras/iris/v12@master from your current project's directory.
  • Upgrade go get -u github.com/kataras/bindata/cmd/bindata if you are going to use its -prefix argument.

Example: file-server/http2push-embedded-gzipped

@xiaozhuai
Copy link
Author

:( Still not work, I have no idea why.
Here is my repo.
https://github.com/xiaozhuai/nmac_mirror

cd backend
go build

And one more thing

	if configuration.HttpsSupport && configuration.RedirectToHttps {
		app.Use(AutoRedirectToHttpsMiddleware(configuration.HttpsPort, configuration.RedirectToHttpsCode))
	}

		............

		go app.Run(
			iris.Addr(fmt.Sprintf("%s:%d", configuration.ListenAddress, configuration.HttpPort)),
			iris.WithoutServerError(iris.ErrServerClosed),
		)

		app.Run(
			iris.TLS(
				fmt.Sprintf("%s:%d", configuration.ListenAddress, configuration.HttpsPort),
				configuration.CertFile,
				configuration.KeyFile,
				func(su *host.Supervisor) {
					su.NoRedirect()
				},
			),
			iris.WithoutServerError(iris.ErrServerClosed),
		)

I can't specify the redirect http server's listen address and port.

So I need to su.NoRedirect() and run a http server and add a middleware to do the redirection.

@kataras
Copy link
Owner

kataras commented Jul 16, 2020

The example runs and works as expected here, let me clone your repository and see what's goin on

@kataras
Copy link
Owner

kataras commented Jul 16, 2020

@xiaozhuai remove the ctx.Header("Vary", "Accept-Encoding") from AssetValidator and it should work:

func AssetsDirOptions() iris.DirOptions {
	return iris.DirOptions{
		IndexName: "/index.html",
		PushTargets: map[string][]string{
			"/": GetPushTargets(),
		},
		Asset:      GzipAsset,
		AssetInfo:  GzipAssetInfo,
		AssetNames: GzipAssetNames,
		AssetValidator: func(ctx iris.Context, name string) bool {
			ctx.Header("Content-Encoding", "gzip")
			return true
		},
	}
}

func GetPushTargets() (pushTargets []string) {
	assetNames := GzipAssetNames()
	for _, name := range assetNames {
		// ignore *.map file
		if !strings.HasSuffix(name, ".map") {
			target := strings.TrimPrefix(name, "public")

			// skip /index.html
			if target != "/index.html" {
				pushTargets = append(pushTargets, target)
			}
		}
	}

	return
}

image

  • Also, instead of func(su... you can just pass iris.TLSNoRedirect.
  • And when your app may require a different Iris Runner it's better to code it like that:
var runner iris.Runner
if configuration.HttpsSupport {
	addr := fmt.Sprintf("%s:%d", configuration.ListenAddress, configuration.HttpsPort)
	runner = iris.TLS(addr, configuration.CertFile, configuration.KeyFile)
} else {
	addr := fmt.Sprintf("%s:%d", configuration.ListenAddress, configuration.HttpPort)
	runner = iris.Addr(addr)
}

app.Run(runner, iris.WithoutServerError(iris.ErrServerClosed)) // all options go here.
  • You don't need iris.TLSNoRedirect. The secondary http: server works but it redirects back to https://0.0.0.0/... because that's the registered virtual host you set, which is not a valid one, try change it to 127.0.0.1 or use a domain instead (map a fake one through your system's hosts file) and it will work just fine.

  • Q: Would you like for PushTargets map[string][]string to accept a PushTargets map[string] *regexp.Regex instead? So you don't have to do the GetPushTargets() thing by your own? After all this field added yesterday, inspirated by this issue of yours

@xiaozhuai
Copy link
Author

Awesome! Thank you very much @kataras ! It works now! You are a very kind man! Thanks for spending time to help me and make the great iris!

For PushTargets, map[string] *regexp.Regex is more advance and complex, map[string][]string is simple. So I suggets we can provide an option of map[string] *regexp.Regex, and keep map[string][]string

@xiaozhuai
Copy link
Author

xiaozhuai commented Jul 17, 2020

By the way, I found this not work on go1.13 and works on go1.14.
Maybe related to

iris/context/compress.go

Lines 184 to 190 in 4e3836e

func AcquireCompressResponseWriter(w ResponseWriter, r *http.Request, level int) (*CompressResponseWriter, error) {
acceptEncoding := r.Header.Values(AcceptEncodingHeaderKey)
if len(acceptEncoding) == 0 {
return nil, ErrResponseNotCompressed
}

Because there is no Values function on Header

@kataras
Copy link
Owner

kataras commented Jul 17, 2020

Yes, the net/http#Header.Values added on go1.14, we can change it to r.Header[AcceptEncodingHeaderKey]. However people SHOULD upgrade Go to latest release, everywhere and always. Go releases don't break compatibility and they contain important fixes and improvements.

@kataras
Copy link
Owner

kataras commented Jul 18, 2020

@xiaozhuai You are very welcomed, I thank YOU. Example using PushTargetsRegexp:

var opts = iris.DirOptions{
	IndexName: "/index.html",
	PushTargetsRegexp: map[string]*regexp.Regexp{
            "/": iris.MatchCommonAssets,
             /* OR a lighter version:
            "/": regexp.MustCompile("((.*).js|(.*).css|(.*).ico)$"), */
	},
	Compress: true,
}

@kataras kataras added this to the v12.2.0 milestone Jul 18, 2020
kataras added a commit that referenced this issue Jul 18, 2020
kataras added a commit that referenced this issue Jul 26, 2020
Former-commit-id: 5a3f626ba0fbcf10be979b5a800eba51e88602cd
kataras added a commit that referenced this issue Jul 26, 2020
Former-commit-id: 940f07fdb1184266dc315441fb91afa5754092fa
kataras added a commit that referenced this issue Jul 26, 2020
Former-commit-id: 60d9b0d77693895fdfbebe83712e9cc1ee3f8f26
kataras added a commit that referenced this issue Jul 26, 2020
rel to: #1562 (comment)


Former-commit-id: 778cf9146b730d424040ea9e259ce6500f53b563
kataras added a commit that referenced this issue Jul 26, 2020
…system

rel to: #1562 (comment)


Former-commit-id: 9e3fb4d71e14dda025a3af86cf210ff72127b716
kataras added a commit that referenced this issue Jul 26, 2020
rel to: #1562 (comment)


Former-commit-id: da04ae0543e9b743cd4989ded5983ae15316a879
@kataras kataras closed this as completed Aug 2, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants