Skip to content

Commit

Permalink
Add Kopia repository connect command (#2662)
Browse files Browse the repository at this point in the history 
* Add safecli dependency

* add new flag implementations based on the safecli package for the Kopia CLI

* apply go fmt

Signed-off-by: pavel.larkin <pavel.larkin@veeam.com>

* Add common Kopia args and flags

* Add Kopia storage core flags

* Add kopia filesystem storage flags

* cleanup storage tests

* Add kopia GCS storage flags

* add gcs flag tests

* Add kopia azure storage flags

* Add kopia s3 and s3 compliant storage flags

* Use test.FlagSuite for storage tests

* Fix typo

* Add kopia CLI repository connect command

* Add kopia CLI repository connect command

* Fix Apply and test.Suit

Signed-off-by: pavel.larkin <pavel.larkin@veeam.com>

* Remove variadic args for Common and Cache flags

Signed-off-by: pavel.larkin <pavel.larkin@veeam.com>

* pkg/kopia/cli/internal/flag is implemented in the safecli@v0.0.4 now

Signed-off-by: pavel.larkin <pavel.larkin@veeam.com>

* Add pkg/kopia/cli package

Signed-off-by: pavel.larkin <pavel.larkin@veeam.com>

* go mod tidy

Signed-off-by: pavel.larkin <pavel.larkin@veeam.com>

* Add Kopia storage helpers

Signed-off-by: pavel.larkin <pavel.larkin@veeam.com>

* Implement Kopia storage Filesystem opts

Signed-off-by: pavel.larkin <pavel.larkin@veeam.com>

* Add (c) headers

Signed-off-by: pavel.larkin <pavel.larkin@veeam.com>

* Remove unused error

Signed-off-by: pavel.larkin <pavel.larkin@veeam.com>

* Reorganize imports

Signed-off-by: pavel.larkin <pavel.larkin@veeam.com>

* Add Kopia GCS storage opts

Signed-off-by: pavel.larkin <pavel.larkin@veeam.com>

* Reorganize imports

Signed-off-by: pavel.larkin <pavel.larkin@veeam.com>

* Add Kopia Azure storage opts

Signed-off-by: pavel.larkin <pavel.larkin@veeam.com>

* Fix gcs test

Signed-off-by: pavel.larkin <pavel.larkin@veeam.com>

* Add Kopia S3 and S3 compliant storage opts

Signed-off-by: pavel.larkin <pavel.larkin@veeam.com>

* Add Kopia S3 and S3 compliant storage opts

Signed-off-by: pavel.larkin <pavel.larkin@veeam.com>

* Cleanup tests

Signed-off-by: pavel.larkin <pavel.larkin@veeam.com>

* Add Kopia repository create command

Signed-off-by: pavel.larkin <pavel.larkin@veeam.com>

* Cleanup

Signed-off-by: pavel.larkin <pavel.larkin@veeam.com>

* Reorganize tests

Signed-off-by: pavel.larkin <pavel.larkin@veeam.com>

* Add Kopia repository connect command

Signed-off-by: pavel.larkin <pavel.larkin@veeam.com>

* Add opts tests

Signed-off-by: pavel.larkin <pavel.larkin@veeam.com>

* Add opts tests

Signed-off-by: pavel.larkin <pavel.larkin@veeam.com>

* Convert common flags from vars to funcs

Signed-off-by: pavel.larkin <pavel.larkin@veeam.com>

* Add safecli dependency

* add new flag implementations based on the safecli package for the Kopia CLI

* apply go fmt

Signed-off-by: pavel.larkin <pavel.larkin@veeam.com>

* Fix Apply and test.Suit

Signed-off-by: pavel.larkin <pavel.larkin@veeam.com>

* pkg/kopia/cli/internal/flag is implemented in the safecli@v0.0.4 now

Signed-off-by: pavel.larkin <pavel.larkin@veeam.com>

* Add pkg/kopia/cli package

Signed-off-by: pavel.larkin <pavel.larkin@veeam.com>

* go mod tidy

Signed-off-by: pavel.larkin <pavel.larkin@veeam.com>

* Update safecli to v0.0.5

Signed-off-by: pavel.larkin <pavel.larkin@veeam.com>

* Update safecli to v0.0.6

Signed-off-by: pavel.larkin <pavel.larkin@veeam.com>

* Fix tests

Signed-off-by: pavel.larkin <pavel.larkin@veeam.com>

* Add Location.IsPointInTypeSupported

Signed-off-by: pavel.larkin <pavel.larkin@veeam.com>

* Add tests for Location.IsPointInTypeSupported

Signed-off-by: pavel.larkin <pavel.larkin@veeam.com>

* Fix s3 options

Signed-off-by: pavel.larkin <pavel.larkin@veeam.com>

* Fix s3 options

Signed-off-by: pavel.larkin <pavel.larkin@veeam.com>

* Fix options to return errors for empty args

Signed-off-by: pavel.larkin <pavel.larkin@veeam.com>

* Fix options to return errors for empty args

Signed-off-by: pavel.larkin <pavel.larkin@veeam.com>

* Fix options to return errors for empty args

Signed-off-by: pavel.larkin <pavel.larkin@veeam.com>

* Support empty prefix

Signed-off-by: pavel.larkin <pavel.larkin@veeam.com>

* Support empty prefix

Signed-off-by: pavel.larkin <pavel.larkin@veeam.com>

* Support empty prefix

Signed-off-by: pavel.larkin <pavel.larkin@veeam.com>

* Support empty prefix

Signed-off-by: pavel.larkin <pavel.larkin@veeam.com>

* Support empty argument for hostname and username options

Signed-off-by: pavel.larkin <pavel.larkin@veeam.com>

* Support PIT option for Azure and S3 only

Signed-off-by: pavel.larkin <pavel.larkin@veeam.com>

* Fix formatting

Signed-off-by: pavel.larkin <pavel.larkin@veeam.com>

* organize imports

Signed-off-by: pavel.larkin <pavel.larkin@veeam.com>

* organize imports

Signed-off-by: pavel.larkin <pavel.larkin@veeam.com>

* organize imports

Signed-off-by: pavel.larkin <pavel.larkin@veeam.com>

* Fix s3 tests

Signed-off-by: pavel.larkin <pavel.larkin@veeam.com>

* Fix test suite

Signed-off-by: pavel.larkin <pavel.larkin@veeam.com>

---------

Signed-off-by: pavel.larkin <pavel.larkin@veeam.com>
  • Loading branch information
@plar
plar authored Mar 5, 2024
1 parent e6c5cb5 commit f456d39
Show file tree
Hide file tree
Showing 5 changed files with 251 additions and 1 deletion.
22 changes: 22 additions & 0 deletions pkg/kopia/cli/internal/kopia_test.go
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
package internal_test

import (
"testing"

"gopkg.in/check.v1"

"github.com/kanisterio/kanister/pkg/kopia/cli/internal"
"github.com/kanisterio/kanister/pkg/kopia/cli/internal/opts"
)

func TestNewKopiaCommand(t *testing.T) { check.TestingT(t) }

type KopiaCommandSuite struct{}

var _ = check.Suite(&KopiaCommandSuite{})

func (s *KopiaCommandSuite) TestNewKopiaCommandSuite(c *check.C) {
cmd, err := internal.NewKopiaCommand(opts.JSON(true))
c.Check(err, check.IsNil)
c.Check(cmd.Build(), check.DeepEquals, []string{"kopia", "--json"})
}
17 changes: 16 additions & 1 deletion pkg/kopia/cli/repository/opts.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@ package repository
import (
"time"

"github.com/go-openapi/strfmt"
"github.com/kanisterio/safecli/command"
"github.com/pkg/errors"

Expand All @@ -33,7 +34,8 @@ import (
var (
cmdRepository = command.NewArgument("repository")

subcmdCreate = command.NewArgument("create")
subcmdCreate = command.NewArgument("create")
subcmdConnect = command.NewArgument("connect")
)

// optHostname creates a new option for the hostname of the repository.
Expand Down Expand Up @@ -89,3 +91,16 @@ func errUnsupportedStorageType(t rs.LocType) command.Applier {
err := errors.Wrapf(cli.ErrUnsupportedStorage, "storage location: %v", t)
return command.NewErrorArgument(err)
}

// optReadOnly creates a new option for the read-only mode of the repository.
func optReadOnly(readOnly bool) command.Applier {
return command.NewOption("--readonly", readOnly)
}

// optPointInTime creates a new option for the point-in-time of the repository.
func optPointInTime(l internal.Location, pit strfmt.DateTime) command.Applier {
if !l.IsPointInTypeSupported() || time.Time(pit).IsZero() {
return command.NewNoopArgument()
}
return command.NewOptionWithArgument("--point-in-time", pit.String())
}
36 changes: 36 additions & 0 deletions pkg/kopia/cli/repository/opts_test.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -15,13 +15,16 @@
package repository

import (
"fmt"
"testing"

"github.com/go-openapi/strfmt"
"github.com/kanisterio/safecli/command"
"github.com/kanisterio/safecli/test"
"gopkg.in/check.v1"

"github.com/kanisterio/kanister/pkg/kopia/cli"
"github.com/kanisterio/kanister/pkg/kopia/cli/internal"
)

func TestRepositoryOptions(t *testing.T) { check.TestingT(t) }
Expand Down Expand Up @@ -76,4 +79,37 @@ var _ = check.Suite(&test.ArgumentSuite{Cmd: "cmd", Arguments: []test.ArgumentTe
Argument: optStorage(locFTP, "repoPathPrefix", nil),
ExpectedErr: cli.ErrUnsupportedStorage,
},
{
Name: "optReadOnly",
Argument: command.NewArguments(
optReadOnly(true),
optReadOnly(false), // no output
),
ExpectedCLI: []string{"cmd", "--readonly"},
},
{
Name: "optPointInTime supported only for azure and s3",
Argument: func() command.Arguments {
locations := []internal.Location{
locFS, // no output
locAzure, // idx: 1
locGCS, // no output
locS3, // idx: 3
locS3Compliant, // idx: 4
}
var args command.Arguments
for idx, l := range locations {
t, _ := strfmt.ParseDateTime(
fmt.Sprintf("2021-02-%02dT01:02:03.000Z", idx),
)
args = append(args, optPointInTime(l, t))
}
return args
}(),
ExpectedCLI: []string{"cmd",
"--point-in-time=2021-02-01T01:02:03.000Z",
"--point-in-time=2021-02-03T01:02:03.000Z",
"--point-in-time=2021-02-04T01:02:03.000Z",
},
},
}})
60 changes: 60 additions & 0 deletions pkg/kopia/cli/repository/repository_connect.go
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,60 @@
// Copyright 2024 The Kanister Authors.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

package repository

import (
"github.com/go-openapi/strfmt"

"github.com/kanisterio/safecli"

"github.com/kanisterio/kanister/pkg/kopia/cli/args"
"github.com/kanisterio/kanister/pkg/kopia/cli/internal"
"github.com/kanisterio/kanister/pkg/kopia/cli/internal/opts"
"github.com/kanisterio/kanister/pkg/log"
)

// ConnectArgs defines the arguments for the `kopia repository connect` command.
type ConnectArgs struct {
args.Common
args.Cache

Hostname string // the hostname of the repository
Username string // the username of the repository
Location map[string][]byte // the location of the repository
RepoPathPrefix string // the prefix of the repository path
ReadOnly bool // connect to a repository in read-only mode
PointInTime strfmt.DateTime // connect to a repository as it was at a specific point in time

Logger log.Logger
}

// Connect creates a new `kopia repository connect ...` command.
func Connect(args ConnectArgs) (*safecli.Builder, error) {
return internal.NewKopiaCommand(
opts.Common(args.Common),
cmdRepository, subcmdConnect,
opts.CheckForUpdates(false),
optReadOnly(args.ReadOnly),
opts.Cache(args.Cache),
optHostname(args.Hostname),
optUsername(args.Username),
optStorage(
args.Location,
args.RepoPathPrefix,
args.Logger,
),
optPointInTime(args.Location, args.PointInTime),
)
}
117 changes: 117 additions & 0 deletions pkg/kopia/cli/repository/repository_connect_test.go
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,117 @@
package repository

import (
"testing"

"github.com/go-openapi/strfmt"
"github.com/kanisterio/safecli"
"gopkg.in/check.v1"

"github.com/kanisterio/kanister/pkg/kopia/cli/internal/test"
)

func TestRepositoryConnectCommand(t *testing.T) { check.TestingT(t) }

// Test Repository Connect command
var _ = check.Suite(test.NewCommandSuite([]test.CommandTest{
{
Name: "repository connect with default retention",
Command: func() (*safecli.Builder, error) {
args := ConnectArgs{
Common: common,
Cache: cache,
Hostname: "test-hostname",
Username: "test-username",
RepoPathPrefix: "test-path/prefix",
Location: locFS,
}
return Connect(args)
},
ExpectedCLI: []string{"kopia",
"--config-file=path/kopia.config",
"--log-dir=cache/log",
"--log-level=error",
"--password=encr-key",
"repository",
"connect",
"--no-check-for-updates",
"--cache-directory=/tmp/cache.dir",
"--content-cache-size-limit-mb=0",
"--metadata-cache-size-limit-mb=0",
"--override-hostname=test-hostname",
"--override-username=test-username",
"filesystem",
"--path=/mnt/data/test-prefix/test-path/prefix/",
},
},
{
Name: "repository connect with ReadOnly",
Command: func() (*safecli.Builder, error) {
args := ConnectArgs{
Common: common,
Cache: cache,
Hostname: "test-hostname",
Username: "test-username",
RepoPathPrefix: "test-path/prefix",
Location: locFS,
ReadOnly: true,
}
return Connect(args)
},
ExpectedCLI: []string{"kopia",
"--config-file=path/kopia.config",
"--log-dir=cache/log",
"--log-level=error",
"--password=encr-key",
"repository",
"connect",
"--no-check-for-updates",
"--readonly",
"--cache-directory=/tmp/cache.dir",
"--content-cache-size-limit-mb=0",
"--metadata-cache-size-limit-mb=0",
"--override-hostname=test-hostname",
"--override-username=test-username",
"filesystem",
"--path=/mnt/data/test-prefix/test-path/prefix/",
},
},
{
Name: "repository connect with PIT and ReadOnly",
Command: func() (*safecli.Builder, error) {
pit, _ := strfmt.ParseDateTime("2021-02-03T01:02:03Z")
args := ConnectArgs{
Common: common,
Cache: cache,
Hostname: "test-hostname",
Username: "test-username",
RepoPathPrefix: "path/prefix",
Location: locS3,
PointInTime: pit,
ReadOnly: true,
}
return Connect(args)
},
ExpectedCLI: []string{"kopia",
"--config-file=path/kopia.config",
"--log-dir=cache/log",
"--log-level=error",
"--password=encr-key",
"repository",
"connect",
"--no-check-for-updates",
"--readonly",
"--cache-directory=/tmp/cache.dir",
"--content-cache-size-limit-mb=0",
"--metadata-cache-size-limit-mb=0",
"--override-hostname=test-hostname",
"--override-username=test-username",
"s3",
"--region=test-region",
"--bucket=test-bucket",
"--endpoint=test-endpoint",
"--prefix=test-prefix/path/prefix/",
"--point-in-time=2021-02-03T01:02:03.000Z",
},
},
}))

0 comments on commit f456d39

Please sign in to comment.