Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

including advanced net, dhcp and dns policies to automated tests #908

Merged
merged 6 commits into from
Mar 2, 2022
Merged

including advanced net, dhcp and dns policies to automated tests #908

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
52053ca
including advanced net, dhcp and dns policies to automated tests
manrodrigues Mar 1, 2022
4b25e48
fix problem on subscription scenarios
manrodrigues Mar 1, 2022
e2dbf0e
removing invalid scenarios
manrodrigues Mar 2, 2022
5e1af8a
Merge branch 'develop' of github.com:ns1labs/orb into tests/policies_…
manrodrigues Mar 2, 2022
e9d5ebf
removing invalid scenarios
manrodrigues Mar 2, 2022
ebaa7c5
removing double spaces
manrodrigues Mar 2, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
49 changes: 27 additions & 22 deletions python-test/docs/development_guide.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,8 @@
| [Provision agent before group (check if agent subscribes to the group)](integration/provision_agent_before_group.md) | ✅ | | 👍 | 👍 |
| [Provision agent after group (check if agent subscribes to the group)](integration/provision_agent_after_group.md) | ✅ | | 👍 | 👍 |
| [Provision agent with tag matching existing group linked to a valid dataset](integration/multiple_agents_subscribed_to_a_group.md) | ✅ | | 👍 | 👍 |
| [Apply multiple policies to a group](integration/apply_multiple_policies.md) | ✅ | | 👍 | 👍 |
| [Apply multiple simple policies to a group](integration/apply_multiple_policies.md) | ✅ | | 👍 | 👍 |
| [Apply multiple advanced policies to a group](integration/apply_multiple_policies.md) | ✅ | | 👍 | 👍 |
| [Apply multiple policies to a group and remove one policy](integration/remove_one_of_multiple_policies.md) | ✅ | | 👍 | 👍 |
| [Apply multiple policies to a group and remove all of them](integration/remove_all_policies.md) | | | | |
| [Apply multiple policies to a group and remove one dataset](integration/remove_one_of_multiple_datasets.md) | ✅ | | 👍 | 👍 |
Expand Down Expand Up @@ -156,27 +157,31 @@
## **POLICIES**


| Policies Scenario | Automated via API | Automated via UI | Smoke | Sanity |
|:--------------------------------------------------------------------------------------------------------------------------:|:-----------------:|:----------------:|:-----:|:------:|
| [Check if total policies on policies' page is correct](policies/check_if_total_policies_on_policies'_page_is_correct.md) | | | | |
| [Create policy with invalid name (regex)](policies/create_policy_with_invalid_name_(regex).md) | | | | 👍 |
| [Create policy with no agent provisioned](policies/create_policy_with_no_agent_provisioned.md) | | | | 👍 |
| [Create policy with duplicate name](policies/create_policy_with_duplicate_name.md) | | | | 👍 |
| [Create policy with description](policies/create_policy_with_description.md) | ✅ | | 👍 | 👍 |
| [Create policy without description](policies/create_policy_without_description.md) | | | | 👍 |
| [Create policy with dhcp handler](policies/create_policy_with_dhcp_handler.md) | ✅ | | 👍 | 👍 |
| [Create policy with dns handler](policies/create_policy_with_dns_handler.md) | ✅ | | 👍 | 👍 |
| [Create policy with net handler](policies/create_policy_with_net_handler.md) | ✅ | | 👍 | 👍 |
| [Create policy with multiple handlers](policies/create_policy_with_multiple_handlers.md) | | | | 👍 |
| [Test policy filters](policies/test_policy_filters.md) | | | |
| [Check policies details](policies/check_policies_details.md) | | | | 👍 |
| [Edit a policy through the details modal](policies/edit_a_policy_through_the_details_modal.md) | | | | 👍 |
| [Edit policy name](policies/edit_policy_name.md) | | | | 👍 |
| [Edit policy description](policies/edit_policy_description.md) | | | | 👍 |
| [Edit policy handler](policies/edit_policy_handler.md) | | | 👍 | 👍 |
| [Check if is possible cancel operations with no change](policies/check_if_is_possible_cancel_operations_with_no_change.md) | | | | |
| [Remove policy using correct name](policies/remove_policy_using_correct_name.md) | ✅ | | 👍 | 👍 |
| [Remove policy using incorrect name](policies/remove_policy_using_incorrect_name.md) | | | | 👍 |
| Policies Scenario | Automated via API | Automated via UI | Smoke | Sanity |
|:--------------------------------------------------------------------------------------------------------------------------------------------------------------------------:|:-----------------:|:----------------:|:-----:|:------:|
| [Check if total policies on policies' page is correct](policies/check_if_total_policies_on_policies'_page_is_correct.md) | | | | |
| [Create policy with invalid name (regex)](policies/create_policy_with_invalid_name_(regex).md) | | | | 👍 |
| [Create policy with no agent provisioned](policies/create_policy_with_no_agent_provisioned.md) | | | | 👍 |
| [Create policy with duplicate name](policies/create_policy_with_duplicate_name.md) | | | | 👍 |
| [Create policy with description](policies/create_policy_with_description.md) | ✅ | | 👍 | 👍 |
| [Create policy without description](policies/create_policy_without_description.md) | | | | 👍 |
| [Create a policy with dns handler, description, host specification, bpf filter, pcap source, only qname suffix and only rcode](policies/create_policy_with_dns_handler.md) | ✅ | | 👍 | 👍 |
| [Create a policy with dns handler, host specification, bpf filter, pcap source, only qname suffix and only rcode](policies/create_policy_with_dns_handler.md) | ✅ | | 👍 | 👍 |
| [Create a policy with dns handler, bpf filter, pcap source, only qname suffix and only rcode](policies/create_policy_with_dns_handler.md) | ✅ | | 👍 | 👍 |
| [Create a policy with dns handler, pcap source, only qname suffix and only rcode](policies/create_policy_with_dns_handler.md) | ✅ | | 👍 | 👍 |
| [Create a policy with dns handler, only qname suffix](policies/create_policy_with_dns_handler.md) | ✅ | | 👍 | 👍 |
| [Create a policy with net handler, description, host specification, bpf filter and pcap source](policies/create_policy_with_net_handler.md) | ✅ | | 👍 | 👍 |
| [Create a policy with dhcp handler, description, host specification, bpf filter and pcap source](policies/create_policy_with_dhcp_handler.md) | ✅ | | 👍 | 👍 |
| [Create policy with multiple handlers](policies/create_policy_with_multiple_handlers.md) | | | | 👍 |
| [Test policy filters](policies/test_policy_filters.md) | | | |
| [Check policies details](policies/check_policies_details.md) | | | | 👍 |
| [Edit a policy through the details modal](policies/edit_a_policy_through_the_details_modal.md) | | | | 👍 |
| [Edit policy name](policies/edit_policy_name.md) | | | | 👍 |
| [Edit policy description](policies/edit_policy_description.md) | | | | 👍 |
| [Edit policy handler](policies/edit_policy_handler.md) | | | 👍 | 👍 |
| [Check if is possible cancel operations with no change](policies/check_if_is_possible_cancel_operations_with_no_change.md) | | | | |
| [Remove policy using correct name](policies/remove_policy_using_correct_name.md) | ✅ | | 👍 | 👍 |
| [Remove policy using incorrect name](policies/remove_policy_using_incorrect_name.md) | | | | 👍 |


---------------------------------
Expand Down
22 changes: 21 additions & 1 deletion python-test/docs/integration/apply_multiple_policies.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,24 @@
## Scenario: apply multiple policies to agents subscribed to a group
## 1- Scenario: apply multiple advanced policies to agents subscribed to a group

Steps:
-
1. Provision an agent with tags
2. Create a group with same tags as agent
3. Create a sink
4. Create multiple advanced policies (with filters, source pcap)
5. Create a dataset linking the group, the sink and one of the policies
6. Create another dataset linking the same group, sink and the other policy

Expected result:
-
- All the policies must be applied to the agent (orb-agent API response)
- The container logs contain the message "policy applied successfully" referred to each policy
- The container logs that were output after all policies have been applied contain the message "scraped metrics for policy" referred to each applied policy
- Referred sink must have active state on response
- Datasets related to all existing policies have validity valid


## 2- Scenario: apply multiple simple policies to agents subscribed to a group

Steps:
-
Expand Down
6 changes: 2 additions & 4 deletions python-test/docs/policies/create_policy_with_dhcp_handler.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,12 +1,10 @@
## Scenario: Create policy with dhcp handler
## Steps:

1 - Create a policy with dhcp handler
## 1 - Create a policy with dhcp handler, description, host specification, bpf filter and pcap source

- REST API Method: POST
- endpoint: /policies/agent/
- header: {authorization:token}


## Expected Result:
### Expected Result:
- Request must have status code 201 (created) and the policy must be created
48 changes: 45 additions & 3 deletions python-test/docs/policies/create_policy_with_dns_handler.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,12 +1,54 @@
## Scenario: Create policy with dns handler
## Steps:

1 - Create a policy with dns handler
## 1 - Create a policy with dns handler, description, host specification, bpf filter, pcap source, only qname suffix and only rcode

- REST API Method: POST
- endpoint: /policies/agent/
- header: {authorization:token}


## Expected Result:
### Expected Result:
- Request must have status code 201 (created) and the policy must be created


## 2 - Create a policy with dns handler, host specification, bpf filter, pcap source, only qname suffix and only rcode

- REST API Method: POST
- endpoint: /policies/agent/
- header: {authorization:token}


### Expected Result:
- Request must have status code 201 (created) and the policy must be created


## 3 - Scenario: Create a policy with dns handler, bpf filter, pcap source, only qname suffix and only rcode

- REST API Method: POST
- endpoint: /policies/agent/
- header: {authorization:token}


### Expected Result:
- Request must have status code 201 (created) and the policy must be created

## 4 - Scenario: Create a policy with dns handler, pcap source, only qname suffix and only rcode

- REST API Method: POST
- endpoint: /policies/agent/
- header: {authorization:token}


### Expected Result:
- Request must have status code 201 (created) and the policy must be created


## 5 - Scenario: Create a policy with dns handler, only qname suffix

- REST API Method: POST
- endpoint: /policies/agent/
- header: {authorization:token}


### Expected Result:
- Request must have status code 201 (created) and the policy must be created
8 changes: 4 additions & 4 deletions python-test/docs/policies/create_policy_with_net_handler.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,12 +1,12 @@
## Scenario: Create policy with net handler
## Steps:

1 - Create a policy with net handler
## 1 - Create a policy with net handler, description, host specification, bpf filter and pcap source

- REST API Method: POST
- endpoint: /policies/agent/
- header: {authorization:token}


## Expected Result:
- Request must have status code 201 (created) and the policy must be created
### Expected Result:
- Request must have status code 201 (created) and the policy must be created

2 changes: 1 addition & 1 deletion python-test/features/datasets.feature
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ Feature: datasets creation
And that an agent with 1 orb tag(s) already exists and is online
And referred agent is subscribed to a group
And that a sink already exists
And that a policy already exists
And that a policy using: handler=dns, description='policy_dns', host_specification=10.0.1.0/24,10.0.2.1/32,2001:db8::/64, bpf_filter_expression=udp port 53, pcap_source=libpcap, only_qname_suffix=[.foo.com/ .example.com], only_rcode=0 already exists
When a new dataset is created using referred group, sink and policy ID
Then the container logs should contain the message "managing agent policy from core" within 10 seconds
And the container logs should contain the message "policy applied successfully" within 10 seconds
Expand Down
7 changes: 5 additions & 2 deletions python-test/features/environment.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,11 @@

def before_scenario(context, scenario):
cleanup_container()
context.containers_id = dict()
context.agent_groups = dict()


def after_scenario(context, scenario):
context.execute_steps('''
Given the Orb user logs in
Then cleanup agents
Expand All @@ -12,8 +17,6 @@ def before_scenario(context, scenario):
Then cleanup policies
Then cleanup datasets
''')
context.containers_id = dict()
context.agent_groups = dict()


def after_feature(context, feature):
Expand Down
Loading