Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

GitHub workflow file. #35

Merged
merged 12 commits into from
Jun 3, 2022
65 changes: 65 additions & 0 deletions .github/workflows/ci.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,65 @@
name: CI
pgtgrly marked this conversation as resolved.
Show resolved Hide resolved

on: [push, pull_request]

jobs:
run_tests:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2

- name: Use Node.js
uses: actions/setup-node@v1

- name: Installing Dredd
run: sudo npm install dredd --global --unsafe-perm=true --allow-root

- name: Installing node-fetch module
run: npm install node-fetch@2

- name: Installing hooks module
run: npm install hooks

- name: Installing https module
run: npm install https

- name: Installing file module
run: npm install fs
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I recommend making a package.json, doing npm install and committing a .lock file, or else this will be a moving target and will break continuously.

Copy link
Contributor Author

@SOPHIA2401 SOPHIA2401 Jun 1, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added package.json and .lock file for all node modules in the commit f58fc02 except the dredd module because of it's broken npm install. I have reached out to the dredd team for the same.


- name: Installing prettytable module
run: python -m pip install -U prettytable
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Recommend adding a Pipfile and using pipenv with a lock, same as for npm.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Corrected in commit f58fc02.


- name: Build the stack
run: docker-compose up -d
shell: bash
working-directory: ./test

- name: Waiting for OpenSearch domain to be up.
run: |
counter=1
for counter in {1..10}
do
if [ $(curl -s -o /dev/null --head -w "%{http_code}" 'https://admin:admin@localhost:9200' -H 'Content-Type:application/json' --insecure -v) -ne 200 ]; then
sleep 30s
else
# Waiting for security plugin to be initialised and become operational.
echo "Waiting for addtional 30 seconds.. for Opensearch domain to be up."
pgtgrly marked this conversation as resolved.
Show resolved Hide resolved
sleep 30s
break
fi
done
if [ $counter -eq 11 ]; then
echo "Unable to connect with OpenSearch URL https://localhost:9200/"
exit 1
fi
shell: bash
working-directory: ./test
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe move this into a .sh file and call that?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Because it's a little piece of code, I think we should keep it in the same file and reduce the unnecessary files.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Created a separate file for the same 1811afc.


- name: Run script file
run: |
# Disabling TLS certificate verfication as hosted on docker.
export NODE_TLS_REJECT_UNAUTHORIZED=0
pgtgrly marked this conversation as resolved.
Show resolved Hide resolved
python driver-code.py
shell: bash
working-directory: ./test

56 changes: 56 additions & 0 deletions test/docker-compose.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,56 @@
version: '3'
services:
opensearch-node1:
image: opensearchproject/opensearch:latest
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We'll want a matrix of versions tested here. Can be added later, but consider it.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure, will work on it.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've created an issue to keep track of it #43.

container_name: opensearch-node1
environment:
- cluster.name=opensearch-cluster
- node.name=opensearch-node1
- discovery.seed_hosts=opensearch-node1,opensearch-node2
- cluster.initial_master_nodes=opensearch-node1,opensearch-node2
- bootstrap.memory_lock=true # along with the memlock settings below, disables swapping
- "OPENSEARCH_JAVA_OPTS=-Xms512m -Xmx512m" # minimum and maximum Java heap size, recommend setting both to 50% of system RAM
ulimits:
memlock:
soft: -1
hard: -1
nofile:
soft: 65536 # maximum number of open files for the OpenSearch user, set to at least 65536 on modern systems
hard: 65536
volumes:
- opensearch-data1:/usr/share/opensearch/data
ports:
- 9200:9200
- 9600:9600 # required for Performance Analyzer
networks:
- opensearch-net

opensearch-node2:
image: opensearchproject/opensearch:latest
container_name: opensearch-node2
environment:
- cluster.name=opensearch-cluster
- node.name=opensearch-node2
- discovery.seed_hosts=opensearch-node1,opensearch-node2
- cluster.initial_master_nodes=opensearch-node1,opensearch-node2
- bootstrap.memory_lock=true
- "OPENSEARCH_JAVA_OPTS=-Xms512m -Xmx512m"
ulimits:
memlock:
soft: -1
hard: -1
nofile:
soft: 65536
hard: 65536
volumes:
- opensearch-data2:/usr/share/opensearch/data
networks:
- opensearch-net

volumes:
opensearch-data1:
opensearch-data2:

networks:
opensearch-net:

104 changes: 104 additions & 0 deletions test/driver-code.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,104 @@
import argparse
import os
from xmlrpc.client import boolean
from prettytable import PrettyTable

class Dredd:
def __init__(self, endpoint, user, path, test_name, test_pass):
if endpoint is not None:
self.endpoint = endpoint
else:
self.endpoint = "https://127.0.0.1:9200"

if user is not None:
self.user = user
else:
self.user = "admin:admin"

if path is not None:
self.path = path
else:
self.path = ""

if test_name is not None:
self.test_name = test_name
else:
self.test_name = ""

if test_pass is not None:
self.test_pass = test_pass
else:
self.test_pass = False

def write_file(self):
file_obj = open("url.txt", mode='w', encoding='utf-8')
text = self.endpoint + " " + self.user
file_obj.write(text)
file_obj.seek(0,0)
file_obj.close()

def dredd_work(self):
# Walking in test directory tree and runing dredd framework.
test_failed_count = 0
test_passed_count = 0
test_failed = []
test_passed = []
test_passes = PrettyTable()
test_fails = PrettyTable()
for dirpath, dirnames, files in os.walk("./models"+self.path):
curr_path = dirpath.split('/')
curr_dir = curr_path[len(curr_path)-1]
if files:
command = "dredd " + dirpath +"/"+ files[1]+ " " + self.endpoint+ " --user=" + self.user + " --hookfiles=" + dirpath + "/" + files[0]
if self.test_name != "":
if self.test_name == curr_dir:
result = os.system(command)
if(result != 0):
test_failed_count = test_failed_count + 1
test_failed.append([curr_dir,dirpath])
else:
test_passed_count = test_passed_count + 1
test_passed.append([curr_dir,dirpath])
else:
result = os.system(command)
if(result != 0):
test_failed_count = test_failed_count + 1
test_failed.append([curr_dir,dirpath])
else:
test_passed_count = test_passed_count + 1
test_passed.append([curr_dir,dirpath])
print("Total number of test cases: ", test_failed_count + test_passed_count )
print("Test Passed: ",test_passed_count)
print("Test failed: ",test_failed_count)
if self.test_pass == True:
test_passes.field_names = ["Model Name", "Directory Path"]
test_passes.add_rows(test_passed)
test_passes.align='l'
print("Results: Test cases passed.",test_passes,sep="\n")

test_fails.field_names = ["Model Name", "Directory Path"]
test_fails.add_rows(test_failed)
test_fails.align='l'
print("Results: Test cases failed.",test_fails,sep="\n")
return len(test_failed)


# Parsing command line arguments:
parser = argparse.ArgumentParser()

parser.add_argument('--endpoint', type=str, required=False)
parser.add_argument('--user', type=str, required=False)
parser.add_argument('--path', type=str, required=False)
parser.add_argument('--testname', type=str, required=False)
parser.add_argument('--testpass', type=bool, required=False)
args = parser.parse_args()

# Check whether default arguments provided by user:
obj = Dredd(args.endpoint, args.user, args.path, args.testname, args.testpass )

# Creating a intermediate file for storing URL.
obj.write_file()

# Running dredd
exit(obj.dredd_work())

55 changes: 55 additions & 0 deletions test/models/_global/ping/OpenSearchModel.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,55 @@
openapi: 3.0.2
info:
title: OpenSearch
version: '2021-11-23'
paths:
/:
get:
description: Returns whether the cluster is running.
operationId: GetPingCluster
responses:
'200':
description: GetPingCluster 200 response
content:
application/json:
schema:
$ref: '#/components/schemas/GetPingClusterResponseContent'
components:
schemas:
GetPingClusterResponseContent:
type: object
properties:
name:
type: string
cluster_name:
type: string
cluster_uuid:
type: string
version:
$ref: '#/components/schemas/intermediateStructure'
tagline:
type: string
intermediateStructure:
type: object
properties:
distribution:
type: string
number:
type: string
build_type:
type: string
build_hash:
type: string
build_date:
type: string
build_snapshot:
type: boolean
nullable: true
lucene_version:
type: string
minimum_wire_compatibility_version:
type: string
minimum_index_compatibility_version:
type: string


Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

newline

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added.

7 changes: 7 additions & 0 deletions test/models/_global/ping/hooks.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
const hooks = require('hooks');

hooks.before("/ > GET > 200 > application/json",function(transactions,done){
transactions.expected.headers['Content-Type'] = "application/json; charset=UTF-8";
done();
});