Skip to content

Commit

Permalink
address comment
Browse files Browse the repository at this point in the history
Signed-off-by: Yun-Tang Hsu <hsuy@vmware.com>
  • Loading branch information
Yun-Tang Hsu committed Oct 3, 2022
1 parent bed014a commit b7ac23b
Show file tree
Hide file tree
Showing 15 changed files with 168 additions and 156 deletions.
26 changes: 13 additions & 13 deletions docs/networkpolicy-recommendation.md
Original file line number Diff line number Diff line change
Expand Up @@ -63,17 +63,17 @@ To see all options and usage examples of these commands, you may run
The `theia policy-recommendation run` command triggers a new policy
recommendation job.
If a new policy recommendation job is created successfully, the
`recommendation ID` of this job will be returned:
`name` of this job will be returned:

```bash
$ theia policy-recommendation run
Successfully created policy recommendation job with ID e998433e-accb-4888-9fc8-06563f073e86
Successfully created policy recommendation job with name pr-e998433e-accb-4888-9fc8-06563f073e86
```

`recommendation ID` is a universally unique identifier ([UUID](
The name of policy recommendation job contains universally unique identifier ([UUID](
https://en.wikipedia.org/wiki/Universally_unique_identifier)) that is
automatically generated when creating a new policy recommendation job. We use
`recommendation ID` to identify different policy recommendation jobs.
this UUID to identify different policy recommendation jobs.

A policy recommendation job may take a few minutes to more than an hour to
complete depending on the number of network flows. By default, this command
Expand All @@ -92,7 +92,7 @@ a previous policy recommendation job.
Given the job created above, we could check its status via:

```bash
$ theia policy-recommendation status e998433e-accb-4888-9fc8-06563f073e86
$ theia policy-recommendation status pr-e998433e-accb-4888-9fc8-06563f073e86
Status of this policy recommendation job is COMPLETED
```

Expand All @@ -110,7 +110,7 @@ written into the Clickhouse database. To retrieve results of the policy
recommendation job created above, run:

```bash
$ theia policy-recommendation retrieve e998433e-accb-4888-9fc8-06563f073e86
$ theia policy-recommendation retrieve pr-e998433e-accb-4888-9fc8-06563f073e86
apiVersion: crd.antrea.io/v1alpha1
kind: ClusterNetworkPolicy
metadata:
Expand Down Expand Up @@ -138,21 +138,21 @@ To apply recommended policies in the cluster, we can save the recommended
policies to a YAML file and apply it using `kubectl`:

```bash
theia policy-recommendation retrieve e998433e-accb-4888-9fc8-06563f073e86 -f recommended_policies.yml
theia policy-recommendation retrieve pr-e998433e-accb-4888-9fc8-06563f073e86 -f recommended_policies.yml
kubectl apply -f recommended_policies.yml
```

### List all policy recommendation jobs

The `theia policy-recommendation list` command lists all undeleted policy
recommendation jobs. `CreationTime`, `CompletionTime`, `ID` and `Status` of each
recommendation jobs. `CreationTime`, `CompletionTime`, `Name` and `Status` of each
policy recommendation job will be displayed in table format. For example:

```bash
$ theia policy-recommendation list
CreationTime CompletionTime ID Status
2022-06-17 18:33:15 N/A 2cf13427-cbe5-454c-b9d3-e1124af7baa2 RUNNING
2022-06-17 18:06:56 2022-06-17 18:08:37 e998433e-accb-4888-9fc8-06563f073e86 COMPLETED
CreationTime CompletionTime Name Status
2022-06-17 18:33:15 N/A pr-2cf13427-cbe5-454c-b9d3-e1124af7baa2 RUNNING
2022-06-17 18:06:56 2022-06-17 18:08:37 pr-e998433e-accb-4888-9fc8-06563f073e86 COMPLETED
```

### Delete a policy recommendation job
Expand All @@ -162,6 +162,6 @@ recommendation job. Please proceed with caution since deletion cannot be
undone. To delete the policy recommendation job created above, run:

```bash
$ theia policy-recommendation delete e998433e-accb-4888-9fc8-06563f073e86
Successfully deleted policy recommendation job with ID e998433e-accb-4888-9fc8-06563f073e86
$ theia policy-recommendation delete pr-e998433e-accb-4888-9fc8-06563f073e86
Successfully deleted policy recommendation job with name: pr-e998433e-accb-4888-9fc8-06563f073e86
```
10 changes: 10 additions & 0 deletions pkg/apis/crd/v1alpha1/types.go
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,16 @@ import (
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
)

const (
NPRecommendationJobInitial string = "Initial"
NPRecommendationJobSubsequent string = "Subsequent"
NPRecommendationStateNew string = "NEW"
NPRecommendationStateScheduled string = "SCHEDULED"
NPRecommendationStateRunning string = "RUNNING"
NPRecommendationStateCompleted string = "COMPLETED"
NPRecommendationStateFailed string = "FAILED"
)

// +genclient
// +k8s:deepcopy-gen:interfaces=k8s.io/apimachinery/pkg/runtime.Object

Expand Down
10 changes: 0 additions & 10 deletions pkg/apis/intelligence/v1alpha1/types.go
Original file line number Diff line number Diff line change
Expand Up @@ -16,16 +16,6 @@ package v1alpha1

import metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"

const (
NPRecommendationJobInitial string = "Initial"
NPRecommendationJobSubsequent string = "Subsequent"
NPRecommendationStateNew string = "NEW"
NPRecommendationStateScheduled string = "SCHEDULED"
NPRecommendationStateRunning string = "RUNNING"
NPRecommendationStateCompleted string = "COMPLETED"
NPRecommendationStateFailed string = "FAILED"
)

// +genclient
// +genclient:nonNamespaced
// +k8s:deepcopy-gen:interfaces=k8s.io/apimachinery/pkg/runtime.Object
Expand Down
30 changes: 15 additions & 15 deletions pkg/theia/commands/policy_recommendation_delete.go
Original file line number Diff line number Diff line change
Expand Up @@ -27,26 +27,26 @@ import (
// policyRecommendationDeleteCmd represents the policy-recommendation delete command
var policyRecommendationDeleteCmd = &cobra.Command{
Use: "delete",
Short: "Delete a policy recommendation Spark job",
Long: `Delete a policy recommendation Spark job by ID.`,
Short: "Delete a policy recommendation job",
Long: `Delete a policy recommendation job by Name.`,
Aliases: []string{"del"},
Args: cobra.RangeArgs(0, 1),
Example: `
Delete the policy recommendation job with ID e998433e-accb-4888-9fc8-06563f073e86
$ theia policy-recommendation delete e998433e-accb-4888-9fc8-06563f073e86
Delete the network policy recommendation job with Name pr-e998433e-accb-4888-9fc8-06563f073e86
$ theia policy-recommendation delete pr-e998433e-accb-4888-9fc8-06563f073e86
`,
RunE: policyRecommendationDelete,
}

func policyRecommendationDelete(cmd *cobra.Command, args []string) error {
recoID, err := cmd.Flags().GetString("id")
prName, err := cmd.Flags().GetString("name")
if err != nil {
return err
}
if recoID == "" && len(args) == 1 {
recoID = args[0]
if prName == "" && len(args) == 1 {
prName = args[0]
}
err = ParseRecommendationID(recoID)
err = ParseRecommendationName(prName)
if err != nil {
return err
}
Expand All @@ -64,14 +64,14 @@ func policyRecommendationDelete(cmd *cobra.Command, args []string) error {
err = theiaClient.Delete().
AbsPath("/apis/intelligence.theia.antrea.io/v1alpha1/").
Resource("networkpolicyrecommendations").
Name("pr-" + recoID).
Name(prName).
Do(context.TODO()).
Error()
if err != nil {
return fmt.Errorf("error when deleting policy recommendation job in Theia Manager: %v", err)
return fmt.Errorf("error when deleting policy recommendation job: %v", err)
}
err = wait.Poll(config.StatusCheckPollInterval, config.StatusCheckPollTimeout, func() (bool, error) {
_, err = getPolicyRecommendationByRecommendationID(theiaClient, recoID)
_, err = getPolicyRecommendationByName(theiaClient, prName)
if err == nil {
return false, nil
} else {
Expand All @@ -81,16 +81,16 @@ func policyRecommendationDelete(cmd *cobra.Command, args []string) error {
if err != nil {
return err
}
fmt.Printf("Successfully deleted policy recommendation job with ID %s\n", recoID)
fmt.Printf("Successfully deleted policy recommendation job with name: %s\n", prName)
return nil
}

func init() {
policyRecommendationCmd.AddCommand(policyRecommendationDeleteCmd)
policyRecommendationDeleteCmd.Flags().StringP(
"id",
"i",
"name",
"",
"ID of the policy recommendation Spark job.",
"",
"Name of the policy recommendation job.",
)
}
8 changes: 4 additions & 4 deletions pkg/theia/commands/policy_recommendation_delete_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ import (
)

func TestPolicyRecommendationDelete(t *testing.T) {
sparkAppID := "e292395c-3de1-11ed-b878-0242ac120002"
nprName := "pr-e292395c-3de1-11ed-b878-0242ac120002"
testCases := []struct {
name string
testServer *httptest.Server
Expand All @@ -40,7 +40,7 @@ func TestPolicyRecommendationDelete(t *testing.T) {
name: "Valid case",
testServer: httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
switch strings.TrimSpace(r.URL.Path) {
case fmt.Sprintf("/apis/intelligence.theia.antrea.io/v1alpha1/networkpolicyrecommendations/pr-%s", sparkAppID):
case fmt.Sprintf("/apis/intelligence.theia.antrea.io/v1alpha1/networkpolicyrecommendations/%s", nprName):
if r.Method == "DELETE" {
w.Header().Set("Content-Type", "application/json")
w.WriteHeader(http.StatusOK)
Expand All @@ -55,7 +55,7 @@ func TestPolicyRecommendationDelete(t *testing.T) {
name: "SparkApplication not found",
testServer: httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
switch strings.TrimSpace(r.URL.Path) {
case fmt.Sprintf("/apis/intelligence.theia.antrea.io/v1alpha1/networkpolicyrecommendations/pr-%s", sparkAppID):
case fmt.Sprintf("/apis/intelligence.theia.antrea.io/v1alpha1/networkpolicyrecommendations/%s", nprName):
http.Error(w, http.StatusText(http.StatusNotFound), http.StatusNotFound)
}
})),
Expand All @@ -75,7 +75,7 @@ func TestPolicyRecommendationDelete(t *testing.T) {
SetupTheiaClientAndConnection = oldFunc
}()
cmd := new(cobra.Command)
cmd.Flags().String("id", sparkAppID, "")
cmd.Flags().String("name", nprName, "")
cmd.Flags().Bool("use-cluster-ip", true, "")
err := policyRecommendationDelete(cmd, []string{})
if tt.expectedErrorMsg == "" {
Expand Down
12 changes: 6 additions & 6 deletions pkg/theia/commands/policy_recommendation_list.go
Original file line number Diff line number Diff line change
Expand Up @@ -26,11 +26,11 @@ import (
// policyRecommendationListCmd represents the policy-recommendation list command
var policyRecommendationListCmd = &cobra.Command{
Use: "list",
Short: "List all policy recommendation Spark jobs",
Long: `List all policy recommendation Spark jobs with name, creation time and status.`,
Short: "List all policy recommendation jobs",
Long: `List all policy recommendation jobs with name, creation time, completion time and status.`,
Aliases: []string{"ls"},
Example: `
List all policy recommendation Spark jobs
List all policy recommendation jobs
$ theia policy-recommendation list
`,
RunE: policyRecommendationList,
Expand Down Expand Up @@ -58,11 +58,11 @@ func policyRecommendationList(cmd *cobra.Command, args []string) error {
Resource("networkpolicyrecommendations").
Do(context.TODO()).Into(nprList)
if err != nil {
return fmt.Errorf("error when getting policy recommendation jobs from Theia Manager: %v", err)
return fmt.Errorf("error when getting policy recommendation job list: %v", err)
}

sparkApplicationTable := [][]string{
{"CreationTime", "CompletionTime", "ID", "Status"},
{"CreationTime", "CompletionTime", "Name", "Status"},
}
for _, npr := range nprList.Items {
if npr.Status.SparkApplication == "" {
Expand All @@ -72,7 +72,7 @@ func policyRecommendationList(cmd *cobra.Command, args []string) error {
[]string{
FormatTimestamp(npr.Status.CreationTimestamp.Time),
FormatTimestamp(npr.Status.CompletionTimestamp.Time),
npr.Status.SparkApplication,
npr.Name,
npr.Status.State,
})
}
Expand Down
15 changes: 10 additions & 5 deletions pkg/theia/commands/policy_recommendation_list_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,7 @@ import (

"github.com/spf13/cobra"
"github.com/stretchr/testify/assert"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"k8s.io/client-go/kubernetes"
restclient "k8s.io/client-go/rest"

Expand All @@ -46,17 +47,21 @@ func TestPolicyRecommendationList(t *testing.T) {
case fmt.Sprintf("/apis/intelligence.theia.antrea.io/v1alpha1/networkpolicyrecommendations"):
nprList := &intelligence.NetworkPolicyRecommendationList{
Items: []intelligence.NetworkPolicyRecommendation{
{Status: intelligence.NetworkPolicyRecommendationStatus{
SparkApplication: "test1",
}},
{
ObjectMeta: metav1.ObjectMeta{
Name: "pr-test1",
},
Status: intelligence.NetworkPolicyRecommendationStatus{
SparkApplication: "test1",
}},
},
}
w.Header().Set("Content-Type", "application/json")
w.WriteHeader(http.StatusOK)
json.NewEncoder(w).Encode(nprList)
}
})),
expectedMsg: []string{"test1"},
expectedMsg: []string{"pr-test1"},
expectedErrorMsg: "",
},
{
Expand All @@ -68,7 +73,7 @@ func TestPolicyRecommendationList(t *testing.T) {
}
})),
expectedMsg: []string{},
expectedErrorMsg: "error when getting policy recommendation jobs from Theia Manager",
expectedErrorMsg: "error when getting policy recommendation job list",
},
}
for _, tt := range testCases {
Expand Down
36 changes: 17 additions & 19 deletions pkg/theia/commands/policy_recommendation_retrieve.go
Original file line number Diff line number Diff line change
Expand Up @@ -24,32 +24,30 @@ import (
// policyRecommendationRetrieveCmd represents the policy-recommendation retrieve command
var policyRecommendationRetrieveCmd = &cobra.Command{
Use: "retrieve",
Short: "Get the recommendation result of a policy recommendation Spark job",
Long: `Get the recommendation result of a policy recommendation Spark job by ID.
Short: "Get the recommendation result of a policy recommendation job",
Long: `Get the recommendation result of a policy recommendation job by name.
It will return the recommended NetworkPolicies described in yaml.`,
Args: cobra.RangeArgs(0, 1),
Example: `
Get the recommendation result with job ID e998433e-accb-4888-9fc8-06563f073e86
$ theia policy-recommendation retrieve --id e998433e-accb-4888-9fc8-06563f073e86
Get the recommendation result with job name pr-e998433e-accb-4888-9fc8-06563f073e86
$ theia policy-recommendation retrieve --name pr-e998433e-accb-4888-9fc8-06563f073e86
Or
$ theia policy-recommendation retrieve e998433e-accb-4888-9fc8-06563f073e86
Use a customized ClickHouse endpoint when connecting to ClickHouse to getting the result
$ theia policy-recommendation retrieve e998433e-accb-4888-9fc8-06563f073e86 --clickhouse-endpoint 10.10.1.1
Use Service ClusterIP when connecting to ClickHouse to getting the result
$ theia policy-recommendation retrieve e998433e-accb-4888-9fc8-06563f073e86 --use-cluster-ip
$ theia policy-recommendation retrieve pr-e998433e-accb-4888-9fc8-06563f073e86
Use Service ClusterIP when getting the result
$ theia policy-recommendation retrieve pr-e998433e-accb-4888-9fc8-06563f073e86 --use-cluster-ip
Save the recommendation result to file
$ theia policy-recommendation retrieve e998433e-accb-4888-9fc8-06563f073e86 --use-cluster-ip --file output.yaml
$ theia policy-recommendation retrieve pr-e998433e-accb-4888-9fc8-06563f073e86 --use-cluster-ip --file output.yaml
`,
RunE: policyRecommendationRetrieve,
}

func init() {
policyRecommendationCmd.AddCommand(policyRecommendationRetrieveCmd)
policyRecommendationRetrieveCmd.Flags().StringP(
"id",
"i",
"name",
"",
"ID of the policy recommendation Spark job.",
"",
"Name of the policy recommendation job.",
)
policyRecommendationRetrieveCmd.Flags().StringP(
"file",
Expand All @@ -60,14 +58,14 @@ func init() {
}

func policyRecommendationRetrieve(cmd *cobra.Command, args []string) error {
recoID, err := cmd.Flags().GetString("id")
prName, err := cmd.Flags().GetString("name")
if err != nil {
return err
}
if recoID == "" && len(args) == 1 {
recoID = args[0]
if prName == "" && len(args) == 1 {
prName = args[0]
}
err = ParseRecommendationID(recoID)
err = ParseRecommendationName(prName)
if err != nil {
return err
}
Expand All @@ -86,9 +84,9 @@ func policyRecommendationRetrieve(cmd *cobra.Command, args []string) error {
if pf != nil {
defer pf.Stop()
}
npr, err := getPolicyRecommendationByRecommendationID(theiaClient, recoID)
npr, err := getPolicyRecommendationByName(theiaClient, prName)
if err != nil {
return fmt.Errorf("error when getting policy recommendation job by using Recommendation ID: %v", err)
return fmt.Errorf("error when getting policy recommendation job by using job name: %v", err)
}
if filePath != "" {
if err := os.WriteFile(filePath, []byte(npr.Status.RecommendationOutcome), 0600); err != nil {
Expand Down
Loading

0 comments on commit b7ac23b

Please sign in to comment.