Skip to content

Commit

Permalink
Feature: Index Schema Field Filtering (#202)
Browse files Browse the repository at this point in the history
* new parameters and schemas added for field based filtering

Signed-off-by: Michael Valdron <mvaldron@redhat.com>

* new types and endpoint signatures to support additional filter parameters

Signed-off-by: Michael Valdron <mvaldron@redhat.com>

* setmap collection type and test cases defined

Signed-off-by: Michael Valdron <mvaldron@redhat.com>

* multi field filtering changes

Signed-off-by: Michael Valdron <mvaldron@redhat.com>

* setmap changes dynamic typing and union

Signed-off-by: Michael Valdron <mvaldron@redhat.com>

* AndFilter and FilterResult

Signed-off-by: Michael Valdron <mvaldron@redhat.com>

* filters use lazy evaluation

Signed-off-by: Michael Valdron <mvaldron@redhat.com>

* StructToMap function

Signed-off-by: Michael Valdron <mvaldron@redhat.com>

* replace setmap (using built-in map) with hashicorp hashset data structure

Signed-off-by: Michael Valdron <mvaldron@redhat.com>

* revert schema changes

Signed-off-by: Michael Valdron <mvaldron@redhat.com>

* use fields instead of reverted getter functions

Signed-off-by: Michael Valdron <mvaldron@redhat.com>

* array fuzzy filter & filter options

Signed-off-by: Michael Valdron <mvaldron@redhat.com>

* attribute names field test cases

Signed-off-by: Michael Valdron <mvaldron@redhat.com>

* simplify type signatures under filter.go

Signed-off-by: Michael Valdron <mvaldron@redhat.com>

* filter resources test cases

Signed-off-by: Michael Valdron <mvaldron@redhat.com>

* fix filterDevfileArrayFuzzy to consider version schema results before filtering out index schema entry

Signed-off-by: Michael Valdron <mvaldron@redhat.com>

* filter by starter projects test cases

Signed-off-by: Michael Valdron <mvaldron@redhat.com>

* filter by link names test cases

Signed-off-by: Michael Valdron <mvaldron@redhat.com>

* fix link getters to return the link label rather than the link itself

Signed-off-by: Michael Valdron <mvaldron@redhat.com>

* filter by command groups test cases

Signed-off-by: Michael Valdron <mvaldron@redhat.com>

* filter by git remote names test cases

Signed-off-by: Michael Valdron <mvaldron@redhat.com>

* add missing string field filter test case definitions

Signed-off-by: Michael Valdron <mvaldron@redhat.com>

* Parameter label sets and getters

Signed-off-by: Michael Valdron <mvaldron@redhat.com>

* filter index parameters

Signed-off-by: Michael Valdron <mvaldron@redhat.com>

* fix icon parameter conflict

Signed-off-by: Michael Valdron <mvaldron@redhat.com>

* revert error returns for FilterDevfileSchemaVersion

Signed-off-by: Michael Valdron <mvaldron@redhat.com>

* add iconUri field to IndexParams

Signed-off-by: Michael Valdron <mvaldron@redhat.com>

* add filter result name field and return error if eval fails

Signed-off-by: Michael Valdron <mvaldron@redhat.com>

* only perform field filtering with IndexParams tied to a field

Signed-off-by: Michael Valdron <mvaldron@redhat.com>

* fix version regex in openapi spec

Signed-off-by: Michael Valdron <mvaldron@redhat.com>

* '/v2index/all?arch=arm64&language=java' call integration test case

Signed-off-by: Michael Valdron <mvaldron@redhat.com>

* REST doc update

Signed-off-by: Michael Valdron <mvaldron@redhat.com>

* update generated source from doc changes

Signed-off-by: Michael Valdron <mvaldron@redhat.com>

* remove minSchemaVersion and maxSchemaVersion from v1 param conversion

Signed-off-by: Michael Valdron <mvaldron@redhat.com>

* fix k8s.io/utils as direct dependency

Signed-off-by: Michael Valdron <mvaldron@redhat.com>

* method not allowed query params removed

Signed-off-by: Michael Valdron <mvaldron@redhat.com>

* string pointer set check function

Signed-off-by: Michael Valdron <mvaldron@redhat.com>

* minVersion & maxVersion filtering with range filtering & unit test fixups

Signed-off-by: Michael Valdron <mvaldron@redhat.com>

* add missing schema version range filters to devfile endpoints

Signed-off-by: Michael Valdron <mvaldron@redhat.com>

* remove fuzzy sets and punctuation removal

Signed-off-by: Michael Valdron <mvaldron@redhat.com>

* /v2index?minVersion=1.1&maxVersion=1.1 integration test cases added

Signed-off-by: Michael Valdron <mvaldron@redhat.com>

* /index?provider=Red%22Hat&resources=.zip integrartion test case added

Signed-off-by: Michael Valdron <mvaldron@redhat.com>

* skip new test cases outside test registry

Signed-off-by: Michael Valdron <mvaldron@redhat.com>

* /index/sample?description=Hello%22World integration test case added

Signed-off-by: Michael Valdron <mvaldron@redhat.com>

* /index/all?description=node integration test case added

Signed-off-by: Michael Valdron <mvaldron@redhat.com>

* /v2index?name=java&default=true integration test case added

Signed-off-by: Michael Valdron <mvaldron@redhat.com>

* /v2index/sample?description=java&default=true integration test case added

Signed-off-by: Michael Valdron <mvaldron@redhat.com>

* /v2index/all?description=java&default=true integration test case added

Signed-off-by: Michael Valdron <mvaldron@redhat.com>

* remove lazy eval from FilterResult

Signed-off-by: Michael Valdron <mvaldron@redhat.com>

* remove unused MakeMockFilterResult

Signed-off-by: Michael Valdron <mvaldron@redhat.com>

* AndFilter params by ref instead of by value

Signed-off-by: Michael Valdron <mvaldron@redhat.com>

* add deprecated query param

Signed-off-by: Michael Valdron <mvaldron@redhat.com>

* add FilterDevfileDeprecated filter function and test cases

Signed-off-by: Michael Valdron <mvaldron@redhat.com>

* update test stacks to test against deprecated parameter

Signed-off-by: Michael Valdron <mvaldron@redhat.com>

* update build index to filter based on deprecated stacks if deprecated parameter is set

Signed-off-by: Michael Valdron <mvaldron@redhat.com>

* integration test cases for deprecated filter parameter

Signed-off-by: Michael Valdron <mvaldron@redhat.com>

---------

Signed-off-by: Michael Valdron <mvaldron@redhat.com>
  • Loading branch information
michael-valdron committed Apr 8, 2024
1 parent d10fc19 commit b950a06
Show file tree
Hide file tree
Showing 29 changed files with 11,140 additions and 661 deletions.
7 changes: 4 additions & 3 deletions index/server/go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -9,14 +9,18 @@ require (
github.com/devfile/registry-support/index/generator v0.0.0
github.com/getkin/kin-openapi v0.117.0
github.com/gin-gonic/gin v1.9.0
github.com/hashicorp/go-set v0.1.13
github.com/hashicorp/go-version v1.4.0
github.com/kylelemons/godebug v0.0.0-20170820004349-d65d576e9348
github.com/mohae/deepcopy v0.0.0-20170929034955-c48cc78d4826
github.com/opencontainers/go-digest v1.0.0
github.com/opencontainers/image-spec v1.1.0
github.com/prometheus/client_golang v1.14.0
golang.org/x/text v0.14.0
gopkg.in/segmentio/analytics-go.v3 v3.1.0
k8s.io/apiextensions-apiserver v0.26.10
k8s.io/apimachinery v0.26.10
k8s.io/utils v0.0.0-20230220204549-a5ecb0141aa5
oras.land/oras-go v1.2.5
)

Expand Down Expand Up @@ -96,7 +100,6 @@ require (
github.com/moby/locker v1.0.1 // indirect
github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd // indirect
github.com/modern-go/reflect2 v1.0.2 // indirect
github.com/mohae/deepcopy v0.0.0-20170929034955-c48cc78d4826 // indirect
github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 // indirect
github.com/openshift/api v0.0.0-20200930075302-db52bc4ef99f // indirect
github.com/pelletier/go-toml/v2 v2.1.1 // indirect
Expand Down Expand Up @@ -143,12 +146,10 @@ require (
gopkg.in/yaml.v2 v2.4.0 // indirect
gopkg.in/yaml.v3 v3.0.1 // indirect
k8s.io/api v0.26.10 // indirect
k8s.io/apiextensions-apiserver v0.26.10 // indirect
k8s.io/client-go v0.26.10 // indirect
k8s.io/klog v1.0.0 // indirect
k8s.io/klog/v2 v2.90.1 // indirect
k8s.io/kube-openapi v0.0.0-20221012153701-172d655c2280 // indirect
k8s.io/utils v0.0.0-20230220204549-a5ecb0141aa5 // indirect
sigs.k8s.io/controller-runtime v0.14.7 // indirect
sigs.k8s.io/json v0.0.0-20220713155537-f223a00ba0e2 // indirect
sigs.k8s.io/structured-merge-diff/v4 v4.2.3 // indirect
Expand Down
3 changes: 3 additions & 0 deletions index/server/go.sum
Original file line number Diff line number Diff line change
Expand Up @@ -306,6 +306,8 @@ github.com/hashicorp/errwrap v1.1.0 h1:OxrOeh75EUXMY8TBjag2fzXGZ40LB6IKw45YeGUDY
github.com/hashicorp/errwrap v1.1.0/go.mod h1:YH+1FKiLXxHSkmPseP+kNlulaMuP3n2brvKWEqk/Jc4=
github.com/hashicorp/go-multierror v1.1.1 h1:H5DkEtf6CXdFp0N0Em5UCwQpXMWke8IA0+lD48awMYo=
github.com/hashicorp/go-multierror v1.1.1/go.mod h1:iw975J/qwKPdAO1clOe2L8331t/9/fmwbPZ6JB6eMoM=
github.com/hashicorp/go-set v0.1.13 h1:k1B5goY3c7OKEzpK+gwAhJexxzAJwDN8kId8YvWrihA=
github.com/hashicorp/go-set v0.1.13/go.mod h1:0/D+R4MFUzJ6XmvjU7liXtznF1eQDxh84GJlhXw+lvo=
github.com/hashicorp/go-version v1.4.0 h1:aAQzgqIrRKRa7w75CKpbBxYsmUoPjzVm1W59ca1L0J4=
github.com/hashicorp/go-version v1.4.0/go.mod h1:fltr4n8CU8Ke44wwGCBoEymUuxUHl09ZGVZPK5anwXA=
github.com/hashicorp/golang-lru v0.5.0/go.mod h1:/m3WP610KZHVQ1SGc6re/UDhFvYD7pJ4Ao+sR/qLZy8=
Expand Down Expand Up @@ -463,6 +465,7 @@ github.com/segmentio/backo-go v0.0.0-20200129164019-23eae7c10bd3 h1:ZuhckGJ10ula
github.com/segmentio/backo-go v0.0.0-20200129164019-23eae7c10bd3/go.mod h1:9/Rh6yILuLysoQnZ2oNooD2g7aBnvM7r/fNVxRNWfBc=
github.com/sergi/go-diff v1.1.0 h1:we8PVUC3FE2uYfodKH/nBHMSetSfHDR6scGdBi+erh0=
github.com/sergi/go-diff v1.1.0/go.mod h1:STckp+ISIX8hZLjrqAeVduY0gWCT9IjLuqbuNXdaHfM=
github.com/shoenig/test v0.6.4 h1:kVTaSd7WLz5WZ2IaoM0RSzRsUD+m8wRR+5qvntpn4LU=
github.com/sirupsen/logrus v1.2.0/go.mod h1:LxeOpSwHxABJmUn/MG1IvRgCAasNZTLOkJPxbbu5VWo=
github.com/sirupsen/logrus v1.4.2/go.mod h1:tLMulIdttU9McNUspp0xgXVQah82FyeX6MwdIuYE2rE=
github.com/sirupsen/logrus v1.6.0/go.mod h1:7uNnSEd1DgxDLC74fIahvMZmmYsHGZGEOFrfsX/uA88=
Expand Down
Loading

0 comments on commit b950a06

Please sign in to comment.