-
-
Notifications
You must be signed in to change notification settings - Fork 5.5k
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Fixes #7608 This PR adds a Go package registry usable with the Go proxy protocol. 
- Loading branch information
Showing
23 changed files
with
751 additions
and
10 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,77 @@ | ||
| --- | ||
| date: "2023-05-10T00:00:00+00:00" | ||
| title: "Go Packages Repository" | ||
| slug: "go" | ||
| weight: 45 | ||
| draft: false | ||
| toc: false | ||
| menu: | ||
| sidebar: | ||
| parent: "packages" | ||
| name: "Go" | ||
| weight: 45 | ||
| identifier: "go" | ||
| --- | ||
|
|
||
| # Go Packages Repository | ||
|
|
||
| Publish Go packages for your user or organization. | ||
|
|
||
| **Table of Contents** | ||
|
|
||
| {{< toc >}} | ||
|
|
||
| ## Publish a package | ||
|
|
||
| To publish a Go package perform a HTTP `PUT` operation with the package content in the request body. | ||
| You cannot publish a package if a package of the same name and version already exists. You must delete the existing package first. | ||
| The package must follow the [documented structure](https://go.dev/ref/mod#zip-files). | ||
|
|
||
| ``` | ||
| PUT https://gitea.example.com/api/packages/{owner}/go/upload | ||
| ``` | ||
|
|
||
| | Parameter | Description | | ||
| | --------- | ----------- | | ||
| | `owner` | The owner of the package. | | ||
|
|
||
| To authenticate to the package registry, you need to provide [custom HTTP headers or use HTTP Basic authentication]({{< relref "doc/development/api-usage.en-us.md#authentication" >}}): | ||
|
|
||
| ```shell | ||
| curl --user your_username:your_password_or_token \ | ||
| --upload-file path/to/file.zip \ | ||
| https://gitea.example.com/api/packages/testuser/go/upload | ||
| ``` | ||
|
|
||
| If you are using 2FA or OAuth use a [personal access token]({{< relref "doc/development/api-usage.en-us.md#authentication" >}}) instead of the password. | ||
|
|
||
| The server responds with the following HTTP Status codes. | ||
|
|
||
| | HTTP Status Code | Meaning | | ||
| | ----------------- | ------- | | ||
| | `201 Created` | The package has been published. | | ||
| | `400 Bad Request` | The package is invalid. | | ||
| | `409 Conflict` | A package with the same name exist already. | | ||
|
|
||
| ## Install a package | ||
|
|
||
| To install a Go package instruct Go to use the package registry as proxy: | ||
|
|
||
| ```shell | ||
| # use latest version | ||
| GOPROXY=https://gitea.example.com/api/packages/{owner}/go go install {package_name} | ||
| # or | ||
| GOPROXY=https://gitea.example.com/api/packages/{owner}/go go install {package_name}@latest | ||
| # use specific version | ||
| GOPROXY=https://gitea.example.com/api/packages/{owner}/go go install {package_name}@{package_version} | ||
| ``` | ||
|
|
||
| | Parameter | Description | | ||
| | ----------------- | ----------- | | ||
| | `owner` | The owner of the package. | | ||
| | `package_name` | The package name. | | ||
| | `package_version` | The package version. | | ||
|
|
||
| If the owner of the packages is private you need to [provide credentials](https://go.dev/ref/mod#private-module-proxy-auth). | ||
|
|
||
| More information about the `GOPROXY` environment variable and how to protect against data leaks can be found in [the documentation](https://go.dev/ref/mod#private-modules). |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,94 @@ | ||
| // Copyright 2023 The Gitea Authors. All rights reserved. | ||
| // SPDX-License-Identifier: MIT | ||
|
|
||
| package goproxy | ||
|
|
||
| import ( | ||
| "archive/zip" | ||
| "fmt" | ||
| "io" | ||
| "path" | ||
| "strings" | ||
|
|
||
| "code.gitea.io/gitea/modules/util" | ||
| ) | ||
|
|
||
| const ( | ||
| PropertyGoMod = "go.mod" | ||
|
|
||
| maxGoModFileSize = 16 * 1024 * 1024 // https://go.dev/ref/mod#zip-path-size-constraints | ||
| ) | ||
|
|
||
| var ( | ||
| ErrInvalidStructure = util.NewInvalidArgumentErrorf("package has invalid structure") | ||
| ErrGoModFileTooLarge = util.NewInvalidArgumentErrorf("go.mod file is too large") | ||
| ) | ||
|
|
||
| type Package struct { | ||
| Name string | ||
| Version string | ||
| GoMod string | ||
| } | ||
|
|
||
| // ParsePackage parses the Go package file | ||
| // https://go.dev/ref/mod#zip-files | ||
| func ParsePackage(r io.ReaderAt, size int64) (*Package, error) { | ||
| archive, err := zip.NewReader(r, size) | ||
| if err != nil { | ||
| return nil, err | ||
| } | ||
|
|
||
| var p *Package | ||
|
|
||
| for _, file := range archive.File { | ||
| nameAndVersion := path.Dir(file.Name) | ||
|
|
||
| parts := strings.SplitN(nameAndVersion, "@", 2) | ||
| if len(parts) != 2 { | ||
| continue | ||
| } | ||
|
|
||
| versionParts := strings.SplitN(parts[1], "/", 2) | ||
|
|
||
| if p == nil { | ||
| p = &Package{ | ||
| Name: strings.TrimSuffix(nameAndVersion, "@"+parts[1]), | ||
| Version: versionParts[0], | ||
| } | ||
| } | ||
|
|
||
| if len(versionParts) > 1 { | ||
| // files are expected in the "root" folder | ||
| continue | ||
| } | ||
|
|
||
| if path.Base(file.Name) == "go.mod" { | ||
| if file.UncompressedSize64 > maxGoModFileSize { | ||
| return nil, ErrGoModFileTooLarge | ||
| } | ||
|
|
||
| f, err := archive.Open(file.Name) | ||
| if err != nil { | ||
| return nil, err | ||
| } | ||
| defer f.Close() | ||
|
|
||
| bytes, err := io.ReadAll(&io.LimitedReader{R: f, N: maxGoModFileSize}) | ||
| if err != nil { | ||
| return nil, err | ||
| } | ||
|
|
||
| p.GoMod = string(bytes) | ||
|
|
||
| return p, nil | ||
| } | ||
| } | ||
|
|
||
| if p == nil { | ||
| return nil, ErrInvalidStructure | ||
| } | ||
|
|
||
| p.GoMod = fmt.Sprintf("module %s", p.Name) | ||
|
|
||
| return p, nil | ||
| } |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,75 @@ | ||
| // Copyright 2023 The Gitea Authors. All rights reserved. | ||
| // SPDX-License-Identifier: MIT | ||
|
|
||
| package goproxy | ||
|
|
||
| import ( | ||
| "archive/zip" | ||
| "bytes" | ||
| "testing" | ||
|
|
||
| "github.com/stretchr/testify/assert" | ||
| ) | ||
|
|
||
| const ( | ||
| packageName = "gitea.com/go-gitea/gitea" | ||
| packageVersion = "v0.0.1" | ||
| ) | ||
|
|
||
| func TestParsePackage(t *testing.T) { | ||
| createArchive := func(files map[string][]byte) *bytes.Reader { | ||
| var buf bytes.Buffer | ||
| zw := zip.NewWriter(&buf) | ||
| for name, content := range files { | ||
| w, _ := zw.Create(name) | ||
| w.Write(content) | ||
| } | ||
| zw.Close() | ||
| return bytes.NewReader(buf.Bytes()) | ||
| } | ||
|
|
||
| t.Run("EmptyPackage", func(t *testing.T) { | ||
| data := createArchive(nil) | ||
|
|
||
| p, err := ParsePackage(data, int64(data.Len())) | ||
| assert.Nil(t, p) | ||
| assert.ErrorIs(t, err, ErrInvalidStructure) | ||
| }) | ||
|
|
||
| t.Run("InvalidNameOrVersionStructure", func(t *testing.T) { | ||
| data := createArchive(map[string][]byte{ | ||
| packageName + "/" + packageVersion + "/go.mod": {}, | ||
| }) | ||
|
|
||
| p, err := ParsePackage(data, int64(data.Len())) | ||
| assert.Nil(t, p) | ||
| assert.ErrorIs(t, err, ErrInvalidStructure) | ||
| }) | ||
|
|
||
| t.Run("GoModFileInWrongDirectory", func(t *testing.T) { | ||
| data := createArchive(map[string][]byte{ | ||
| packageName + "@" + packageVersion + "/subdir/go.mod": {}, | ||
| }) | ||
|
|
||
| p, err := ParsePackage(data, int64(data.Len())) | ||
| assert.NotNil(t, p) | ||
| assert.NoError(t, err) | ||
| assert.Equal(t, packageName, p.Name) | ||
| assert.Equal(t, packageVersion, p.Version) | ||
| assert.Equal(t, "module gitea.com/go-gitea/gitea", p.GoMod) | ||
| }) | ||
|
|
||
| t.Run("Valid", func(t *testing.T) { | ||
| data := createArchive(map[string][]byte{ | ||
| packageName + "@" + packageVersion + "/subdir/go.mod": []byte("invalid"), | ||
| packageName + "@" + packageVersion + "/go.mod": []byte("valid"), | ||
| }) | ||
|
|
||
| p, err := ParsePackage(data, int64(data.Len())) | ||
| assert.NotNil(t, p) | ||
| assert.NoError(t, err) | ||
| assert.Equal(t, packageName, p.Name) | ||
| assert.Equal(t, packageVersion, p.Version) | ||
| assert.Equal(t, "valid", p.GoMod) | ||
| }) | ||
| } |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.