Microservice test: Does the CNF have a reasonable container image size?

[lixuna]

Acceptance Criteria

Microservice test: Does the CNF have a reasonable container image size?

EPIC: CNF Conformance Test Suite

Release: MVP

Test Category: Microservice

Type of test (static or runtime) = Runtime

• example code: POC ISSUE_NUMBER > N/A

---

Environment set up tasks:

Upstream tool set up tasks: (conformance suite + upstream tools)

• N/A

CNF setup Tasks

Sample CNF tasks:

Code implementation tasks:

• TBD

Documentation tasks:

• Add Microservice to Test Categories md
• Add Microservice section and new test to Pseudo Code md if needed
• Add Microservice section and new test to USAGE md if needed
• Add Microservice section and new tests to installation instructions if needed

QA tasks

Dev Review:

• walk through A/C
• do you get the expected result?
• if yes,
  • move to Needs Peer Review column
  • create Pull Request and follow check list
  • Assign 1 or more people for peer review
• if no, document what additional tasks will be needed

Peer review:

• walk through A/C
• do you get the expected result?
• if yes,
  • move to Reviewer Approved column
  • Approve pull request
• if no,
  • document what did not go as expected, including error messages and screenshots (if possible)
  • Add comment to pull request
  • request changes to pull request

[lixuna]

Level of effort in pts: 0,1,2,3,5,8,13,21: 5pts

@denverwilliams 3
@williscool 3
@wavell 5
@taylor 5
@lixuna 5

[lixuna]

Acceptance Criteria:

Acceptance Criteria for peer review:

--

Common steps

• Find or create a K8s cluster to use
  • Steps for creating will be in another ticket
  • Talk with team for access to an existing cluster
• Set the KUBECONFIG environment variable for accessing the target K8s cluster
• As a developer, Pull down latest version of CNF conformance test code from develop branch
• As a peer reviewer, Pull down latest version of CNF conformance test code from master branch
• Run shards install
• Run crystal spec
• I expect to see PASSED for the tests

When using the test crystal src/cnf-conformance.cr reasonable_image_size command the sample "good" CNF should PASS test successfully:

• Run crystal src/cnf-conformance.cr setup
• Run crystal src/cnf-conformance.cr sample_coredns_setup
• Run crystal src/cnf-conformance.cr reasonable_image_size
• I expect to see the output and results from the test
• I expect to see PASSED for the test

When using the test crystal spec --tag reasonable_image_size command the sample "good" CNF should PASS and the "bad" oversized image example should FAIL test:

• Run crystal spec --tag reasonable_image_size
• I expect to see the output and results from the test
• I expect to see "PASSED: compliant image size" (for example)
• I expect to see "FAILURE: non-compliant image size" (for example)

I would expect the CNF to pass when I run the entire test category suite (microservice)

• Run crystal src/cnf-conformance.cr setup
• Run crystal src/cnf-conformance.cr sample_coredns_setup
• Run crystal src/cnf-conformance.cr microservice (this is a new test category)
• I expect to see the output and results from all tests in the test category
• I expect to see PASSED for the test category

I would expect the CNF to pass when I run the entire test suite (all)

• Follow common steps above
• Run crystal src/cnf-conformance.cr all
• I expect to see the output from the test
• I expect to see PASSED for the test

[taylor]

@wavell some emoji:

General

PASS: ✔️

FAIL: ❌

Specific to this test:

PASS:

• ✔️ Reasonable container image size🏋️🎈

FAIL:

• ❌Unreasonable container image size🏋️‼️ 🙅

Reference:

• https://emojipedia.org/person-lifting-weights/
• https://emojipedia.org/balloon/
• https://emojipedia.org/double-exclamation-mark/
• https://emojipedia.org/person-gesturing-no/

[taylor]

Recommend having the status at the start and other emojis after. Example Kind:

Creating cluster "test1" ...
✓ Ensuring node image (kindest/node:v1.17.0) 🖼
✓ Preparing nodes 📦
✓ Writing configuration 📜
⢄⡱ Starting control-plane 🕹️

[taylor]

Future improvement:

• do not download/deploy the 5Gb+ image. Need to update helm manifest code to use helm dry-run (and debug) to view computed helm chart vs installing. => https://helm.sh/docs/chart_template_guide/debugging/

[lixuna]

@williscool this is a new test category. Has it been added to the documentation?

in the description:

Documentation tasks:

• Update Test Categories md if needed
• Update Pseudo Code md if needed
• Update USAGE md if needed
• Update installation instructions if needed

[williscool]

☺ ➔  crystal src/cnf-conformance.cr reasonable_image_size
PASSED: Image size is good

[williscool]

micro_size: 13398749
PASSED: Image size is good


...

micro_size: 7230926577
FAILURE: Image size too large
.

Finished in 1:41 minutes
2 examples, 0 failures, 0 errors, 0 pending

[williscool]

Final score: 54
Results have been saved to cnf-conformance-results-20200507-161413-213.yml

---
name: cnf conformance
status:
points:
items:
- name: privileged
  status: passed
  points: 5
- name: increase_capacity
  status: passed
  points: 10
- name: decrease_capacity
  status: passed
  points: 10
- name: ip_addresses
  status: failed
  points: -1
- name: liveness
  status: passed
  points: 5
- name: readiness
  status: passed
  points: 5
- name: install_script_helm
  status: passed
  points: 5
- name: helm_chart_valid
  status: passed
  points: 5
- name: helm_chart_published
  status: passed
  points: 5
- name: reasonable_image_size
  status: passed
  points: 5
- name: reasonable_startup_time
  status: passed
  points: 5