Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

test: adding unit tests for dapr and updating dapr sdk version #2846

Merged
merged 10 commits into from
Jul 24, 2023
11 changes: 6 additions & 5 deletions go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -10,11 +10,12 @@ require (
contrib.go.opencensus.io/exporter/ocagent v0.7.0
contrib.go.opencensus.io/exporter/prometheus v0.4.2
contrib.go.opencensus.io/exporter/stackdriver v0.13.14
github.com/dapr/go-sdk v1.6.0
github.com/dapr/go-sdk v1.8.0
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We may need to start documenting dapr version compatibilities for each version of GK. wdyt?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, I could add it to doc PR 🤔

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you have any specific place in mind for this information?

Copy link
Member

@ritazh ritazh Jul 24, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For docs, might be good to add to the pubsub dapr provider docs. But we need to remember to update that doc whenever this dependency is bumped. Not very ideal.

For the daprClient, is there any minimum required version we need to test? Does the dapr sdk return its own version? If so, might be good to include that in the log as the daprClient is created.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As follow up when we implement batch publishing, 1.8.0 will be the minimum required version, and dapr 1.11 will be the minimum compatible version.

Does the dapr sdk return its own version?

yes it does, but I am not sure how user can figure out which dapr runtime to install before hand if daprClient version information is going to be logged when user has installed all prerequisites and trying to initiate a connection using dapr.

Can we just say in the provider doc that, "current dapr sdk version is [link to gk go mod file]" and point to dapr-version for more information?

github.com/davecgh/go-spew v1.1.1
github.com/dominikbraun/graph v0.16.2
github.com/go-logr/logr v1.2.4
github.com/go-logr/zapr v1.2.4
github.com/golang/protobuf v1.5.3
github.com/google/go-cmp v0.5.9
github.com/google/uuid v1.3.0
github.com/onsi/gomega v1.27.7
Expand All @@ -23,14 +24,16 @@ require (
github.com/pkg/errors v0.9.1
github.com/prometheus/client_golang v1.16.0
github.com/spf13/cobra v1.7.0
github.com/stretchr/testify v1.8.2
github.com/stretchr/testify v1.8.3
go.opencensus.io v0.24.0
go.uber.org/automaxprocs v1.5.2
go.uber.org/zap v1.24.0
golang.org/x/net v0.11.0
golang.org/x/oauth2 v0.7.0
golang.org/x/sync v0.2.0
golang.org/x/time v0.3.0
google.golang.org/grpc v1.56.1
google.golang.org/protobuf v1.30.0
gopkg.in/yaml.v2 v2.4.0
gopkg.in/yaml.v3 v3.0.1
k8s.io/api v0.27.2
Expand Down Expand Up @@ -75,6 +78,7 @@ require (
github.com/felixge/httpsnoop v1.0.3 // indirect
github.com/fsnotify/fsnotify v1.6.0 // indirect
github.com/ghodss/yaml v1.0.0 // indirect
github.com/go-chi/chi/v5 v5.0.8 // indirect
github.com/go-ini/ini v1.67.0 // indirect
github.com/go-kit/log v0.2.1 // indirect
github.com/go-logfmt/logfmt v0.5.1 // indirect
Expand All @@ -86,7 +90,6 @@ require (
github.com/gogo/protobuf v1.3.2 // indirect
github.com/golang/glog v1.1.1 // indirect
github.com/golang/groupcache v0.0.0-20210331224755-41bb18bfe9da // indirect
github.com/golang/protobuf v1.5.3 // indirect
github.com/google/cel-go v0.12.6 // indirect
github.com/google/gnostic v0.5.7-v3refs // indirect
github.com/google/gofuzz v1.2.0 // indirect
Expand Down Expand Up @@ -146,8 +149,6 @@ require (
google.golang.org/api v0.114.0 // indirect
google.golang.org/appengine v1.6.7 // indirect
google.golang.org/genproto v0.0.0-20230410155749-daa745c078e1 // indirect
google.golang.org/grpc v1.56.1 // indirect
google.golang.org/protobuf v1.30.0 // indirect
gopkg.in/inf.v0 v0.9.1 // indirect
k8s.io/apiserver v0.27.2 // indirect
k8s.io/component-base v0.27.2 // indirect
Expand Down
10 changes: 6 additions & 4 deletions go.sum
Original file line number Diff line number Diff line change
Expand Up @@ -362,8 +362,8 @@ github.com/d2g/dhcp4 v0.0.0-20170904100407-a1d1b6c41b1c/go.mod h1:Ct2BUK8SB0YC1S
github.com/d2g/dhcp4client v1.0.0/go.mod h1:j0hNfjhrt2SxUOw55nL0ATM/z4Yt3t2Kd1mW34z5W5s=
github.com/d2g/dhcp4server v0.0.0-20181031114812-7d4a0a7f59a5/go.mod h1:Eo87+Kg/IX2hfWJfwxMzLyuSZyxSoAug2nGa1G2QAi8=
github.com/d2g/hardwareaddr v0.0.0-20190221164911-e7d9fbe030e4/go.mod h1:bMl4RjIciD2oAxI7DmWRx6gbeqrkoLqv3MV0vzNad+I=
github.com/dapr/go-sdk v1.6.0 h1:jg5A2khSCHF8bGZsig5RWN/gD0jjitszc2V6Uq2pPdY=
github.com/dapr/go-sdk v1.6.0/go.mod h1:KLQBltoD9K0w5hKTihdcyg9Epob9gypwL5dYcQzPro4=
github.com/dapr/go-sdk v1.8.0 h1:OEleeL3zUTqXxIZ7Vkk3PClAeCh1g8sZ1yR2JFZKfXM=
github.com/dapr/go-sdk v1.8.0/go.mod h1:MBcTKXg8PmBc8A968tVWQg1Xt+DZtmeVR6zVVVGcmeA=
github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
Expand Down Expand Up @@ -464,6 +464,8 @@ github.com/getsentry/raven-go v0.2.0/go.mod h1:KungGk8q33+aIAZUIVWZDr2OfAEBsO49P
github.com/ghodss/yaml v0.0.0-20150909031657-73d445a93680/go.mod h1:4dBDuWmgqj2HViK6kFavaiC9ZROes6MMH2rRYeMEF04=
github.com/ghodss/yaml v1.0.0 h1:wQHKEahhL6wmXdzwWG11gIVCkOv05bNOh+Rxn0yngAk=
github.com/ghodss/yaml v1.0.0/go.mod h1:4dBDuWmgqj2HViK6kFavaiC9ZROes6MMH2rRYeMEF04=
github.com/go-chi/chi/v5 v5.0.8 h1:lD+NLqFcAi1ovnVZpsnObHGW4xb4J8lNmoYVfECH1Y0=
github.com/go-chi/chi/v5 v5.0.8/go.mod h1:DslCQbL2OYiznFReuXYUmQ2hGd1aDpCnlMNITLSKoi8=
github.com/go-gl/glfw v0.0.0-20190409004039-e6da0acd62b1/go.mod h1:vR7hzQXu2zJy9AVAgeJqvqgH9Q5CA+iKCZ2gyEVpxRU=
github.com/go-gl/glfw/v3.3/glfw v0.0.0-20191125211704-12ad95a8df72/go.mod h1:tQ2UAYgL5IevRw8kRxooKSPJfGvJ9fJQFa0TUsXzTg8=
github.com/go-gl/glfw/v3.3/glfw v0.0.0-20200222043503-6f7a984d4dc4/go.mod h1:tQ2UAYgL5IevRw8kRxooKSPJfGvJ9fJQFa0TUsXzTg8=
Expand Down Expand Up @@ -1161,8 +1163,8 @@ github.com/stretchr/testify v1.7.0/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/
github.com/stretchr/testify v1.7.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
github.com/stretchr/testify v1.8.0/go.mod h1:yNjHg4UonilssWZ8iaSj1OCr/vHnekPRkoO+kdMU+MU=
github.com/stretchr/testify v1.8.1/go.mod h1:w2LPCIKwWwSfY2zedu0+kehJoqGctiVI29o6fzry7u4=
github.com/stretchr/testify v1.8.2 h1:+h33VjcLVPDHtOdpUCuF+7gSuG3yGIftsP1YvFihtJ8=
github.com/stretchr/testify v1.8.2/go.mod h1:w2LPCIKwWwSfY2zedu0+kehJoqGctiVI29o6fzry7u4=
github.com/stretchr/testify v1.8.3 h1:RP3t2pwF7cMEbC1dqtB6poj3niw/9gnV4Cjg5oW5gtY=
github.com/stretchr/testify v1.8.3/go.mod h1:sz/lmYIOXD/1dqDmKjjqLyZ2RngseejIcXlSw2iwfAo=
github.com/stvp/go-udp-testing v0.0.0-20201019212854-469649b16807/go.mod h1:7jxmlfBCDBXRzr0eAQJ48XC1hBu1np4CS5+cHEYfwpc=
github.com/subosito/gotenv v1.2.0/go.mod h1:N0PQaV/YGNqwC0u51sEeR/aUtSLEXKX9iv69rRypqCw=
github.com/syndtr/gocapability v0.0.0-20170704070218-db04d3cc01c8/go.mod h1:hkRG7XYTFWNJGYcbNJQlaLq0fg1yr4J4t/NcTQtrfww=
Expand Down
26 changes: 4 additions & 22 deletions pkg/audit/manager.go
Original file line number Diff line number Diff line change
Expand Up @@ -245,16 +245,12 @@ func (am *Manager) audit(ctx context.Context) error {
var res []Result
am.log.Info("Auditing from cache")
res, errs := am.auditFromCache(ctx)

am.log.Info("Audit from cache results", "violations", len(res))
for _, err := range errs {
am.log.Error(err, "Auditing")
}

err := am.addAuditResponsesToUpdateLists(updateLists, res, totalViolationsPerConstraint, totalViolationsPerEnforcementAction, timestamp)
if errs != nil {
return err
}
am.addAuditResponsesToUpdateLists(updateLists, res, totalViolationsPerConstraint, totalViolationsPerEnforcementAction, timestamp)
} else {
am.log.Info("Auditing via discovery client")
err := am.auditResources(ctx, constraintsGVKs, updateLists, totalViolationsPerConstraint, totalViolationsPerEnforcementAction, timestamp)
Expand Down Expand Up @@ -568,7 +564,6 @@ func (am *Manager) reviewObjects(ctx context.Context, kind string, folderCount i
totalViolationsPerEnforcementAction map[util.EnforcementAction]int64,
timestamp string,
) error {
var errs []error
for i := 0; i < folderCount; i++ {
// cache directory structure:
// apiCacheDir/kind_folderIndex/fileIndex
Expand Down Expand Up @@ -652,18 +647,10 @@ func (am *Manager) reviewObjects(ctx context.Context, kind string, folderCount i

if len(resp.Results()) > 0 {
results := ToResults(&augmentedObj.Object, resp)
err = am.addAuditResponsesToUpdateLists(updateLists, results, totalViolationsPerConstraint, totalViolationsPerEnforcementAction, timestamp)
if err != nil {
// updated to not return err immediately
errs = append(errs, err)
continue
}
am.addAuditResponsesToUpdateLists(updateLists, results, totalViolationsPerConstraint, totalViolationsPerEnforcementAction, timestamp)
}
}
}
if len(errs) > 0 {
return mergeErrors(errs)
}
return nil
}

Expand Down Expand Up @@ -780,8 +767,7 @@ func (am *Manager) addAuditResponsesToUpdateLists(
totalViolationsPerConstraint map[util.KindVersionName]int64,
totalViolationsPerEnforcementAction map[util.EnforcementAction]int64,
timestamp string,
) error {
var errs error
) {
for _, r := range res {
key := util.GetUniqueKey(*r.Constraint)
totalViolationsPerConstraint[key]++
Expand Down Expand Up @@ -817,17 +803,13 @@ func (am *Manager) addAuditResponsesToUpdateLists(
if *pubsubController.PubsubEnabled {
err := am.pubsubSystem.Publish(context.Background(), *auditConnection, *auditChannel, violationMsg(r.Constraint, ea, gvk, namespace, name, r.Msg, details, r.obj.GetLabels(), timestamp))
if err != nil {
errs = errors.Join(errs, err)
am.log.Error(err, "pubsub audit Publishing")
}
}
if *emitAuditEvents {
emitEvent(r.Constraint, timestamp, ea, gvk, namespace, name, rv, r.Msg, am.gkNamespace, uid, am.eventRecorder)
}
}
if errs != nil {
return fmt.Errorf("encountered errors in publishing messages, errors: %w", errs)
}
return nil
}

func (am *Manager) writeAuditResults(ctx context.Context, constraintsGVKs []schema.GroupVersionKind, updateLists map[util.KindVersionName][]updateListEntry, timestamp string, totalViolations map[util.KindVersionName]int64) {
Expand Down
2 changes: 2 additions & 0 deletions pkg/controller/pubsub/pubsub_config_controller.go
Original file line number Diff line number Diff line change
Expand Up @@ -137,10 +137,12 @@ func (r *Reconciler) Reconcile(ctx context.Context, request reconcile.Request) (
if err != nil {
return reconcile.Result{}, err
}

err = r.system.UpsertConnection(ctx, config, request.Name, cfg.Data["provider"])
if err != nil {
return reconcile.Result{}, err
}

log.Info("Connection upsert successful", "name", request.Name, "provider", cfg.Data["provider"])
return reconcile.Result{}, nil
}
2 changes: 1 addition & 1 deletion pkg/pubsub/dapr/dapr.go
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,7 @@ func (r *Dapr) CloseConnection() error {
return nil
}

func (r *Dapr) UpdateConnection(ctx context.Context, config interface{}) error {
func (r *Dapr) UpdateConnection(_ context.Context, config interface{}) error {
var cfg ClientConfig
m, ok := config.(map[string]interface{})
if !ok {
Expand Down
113 changes: 113 additions & 0 deletions pkg/pubsub/dapr/dapr_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -2,12 +2,26 @@ package dapr

import (
"context"
"os"
"testing"

"github.com/open-policy-agent/gatekeeper/v3/pkg/pubsub/connection"
"github.com/stretchr/testify/assert"
)

var testClient connection.Connection

func TestMain(m *testing.M) {
c, f := FakeConnection()
testClient = c
r := m.Run()
f()

if r != 0 {
os.Exit(r)
}
}

func TestNewConnection(t *testing.T) {
tests := []struct {
name string
Expand Down Expand Up @@ -36,3 +50,102 @@ func TestNewConnection(t *testing.T) {
})
}
}

func TestDapr_Publish(t *testing.T) {
ctx := context.Background()

type args struct {
ctx context.Context
data interface{}
topic string
}

tests := []struct {
name string
args args
wantErr bool
}{
{
name: "test publish",
args: args{
ctx: ctx,
data: map[string]interface{}{
"test": "test",
},
topic: "test",
},
wantErr: false,
},
{
name: "test publish without data",
args: args{
ctx: ctx,
data: nil,
topic: "test",
},
wantErr: false,
},
{
name: "test publish without topic",
args: args{
ctx: ctx,
data: map[string]interface{}{
"test": "test",
},
topic: "",
},
wantErr: true,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
r := testClient
if err := r.Publish(tt.args.ctx, tt.args.data, tt.args.topic); (err != nil) != tt.wantErr {
t.Errorf("Dapr.Publish() error = %v, wantErr %v", err, tt.wantErr)
}
})
}
}

func TestDapr_UpdateConnection(t *testing.T) {
tests := []struct {
name string
config interface{}
wantErr bool
}{
{
name: "test update connection",
config: map[string]interface{}{
"component": "foo",
},
wantErr: false,
},
{
name: "test update connection with invalid config",
config: map[string]interface{}{
"foo": "bar",
},
wantErr: true,
},
{
name: "test update connection with nil config",
config: nil,
wantErr: true,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
r := testClient
if err := r.UpdateConnection(context.Background(), tt.config); (err != nil) != tt.wantErr {
t.Errorf("Dapr.UpdateConnection() error = %v, wantErr %v", err, tt.wantErr)
}
if !tt.wantErr {
cmp, ok := tt.config.(map[string]interface{})["component"].(string)
assert.True(t, ok)
tmp, ok := r.(*Dapr)
assert.True(t, ok)
assert.Equal(t, cmp, tmp.pubSubComponent)
}
})
}
}
Loading
Loading