Week 7 notes

notes/lecture06_monitoring.md

@@ -9,10 +9,10 @@ The monitoring system we want to implement is Prometheus. Reasons for choosing P
  - Prometheus can send us alerts if any metrics go over any thresholds that we determine. This can help us catch problems early.
 
 We are going to implement **Application monitoring** and **Infrastructure monitoring**. The metrics we want to measure are:
- - Number of tweet requests per hour - This will help us determining the amount of load our application is under. We will be able to see when something critical is happening to our tweeting system, which is arguably the most important feature in our application.
- - How long it takes to connect to the application - By constantly monitoring (e.g. every 15 minutes) how long it takes to connect to our service, we can determine when our application is unreachable, and roughly for how long it was unreachable. We can also determine whether we have made any changes which makes our application slower to load.
- - Amount of registrations per hour - This will help us determine how many new users we receive. If we suddenly receive a lot of tweet request, logins or some kind of errors, it would be handy to be able to see whether we recently have received a lot of registration requests.
- - Amount of logins per hour - This will help us determine how many logins we receive. If something happens to our service, it would be handy to be able to see whether we have recently received a lot of login requests.
+ - **Number of tweet requests per hour** - This will help us determining the amount of load our application is under. We will be able to see when something critical is happening to our tweeting system, which is arguably the most important feature in our application.
+ - **How long it takes to connect to the application** - By constantly monitoring (e.g. every 15 minutes) how long it takes to connect to our service, we can determine when our application is unreachable, and roughly for how long it was unreachable. We can also determine whether we have made any changes which makes our application slower to load.
+ - **Error codes** - This metric shows us the total amount of errors which the system reports. This metric can be used to determine whether everything runs as expected. If the dashboard displays an unusual amount of errors, this might be a hint that something is going on with the system.
+ - **Hardware load** - This metric reports system load, CPU and memory usage, disk I/O and amount of network transmit and receive requests. This can be useful for determining whether everything runs as expected.
 
 In order to add Prometheus and Grafana to our application, we added the following to our `docker-compose.yml` file:
 ``` yaml
@@ -25,6 +25,7 @@ prometheus:
     container_name: prometheus
     networks:
       - prometheus-network
+      - monitoring
     ports:
       - "9090:9090"
     volumes:
@@ -42,11 +43,30 @@ prometheus:
     container_name: grafana
     ports:
       - "3000:3000"
+    environment:
+      GF_SECURITY_ADMIN_USER: ${GRAFANA_USERNAME}
+      GF_SECURITY_ADMIN_PASSWORD: ${GRAFANA_PASSWORD}
     volumes:
       - grafana-data:/var/lib/grafana
       - ./grafana/provisioning/:/etc/grafana/provisioning/
       - ./grafana/dashboards:/var/lib/grafana/dashboards
     restart: unless-stopped
+  node_exporter:
+    image: prom/node-exporter:latest
+    container_name: node-exporter
+    restart: unless-stopped
+    volumes:
+      - /proc:/host/proc:ro
+      - /sys:/host/sys:ro
+      - /:/rootfs:ro
+    command:
+      - "--path.procfs=/host/proc"
+      - "--path.rootfs=/rootfs"
+      - "--path.sysfs=/host/sys"
+      - "--collector.filesystem.mount-points-exclude=^/(sys|proc|dev|host|etc)($$|/)"
+    networks:
+      - monitoring
+
 
 volumes:
   postgres-db:

notes/lecture07_software_quality.md

@@ -0,0 +1,37 @@
+# Lecture 7: 
+[Week 7](https://github.com/itu-devops/lecture_notes/blob/master/sessions/session_07/README_TASKS.md)
+
+## Add tests to your CI chain
+Tests have been added to our CI/CD setup. Our CI/CD actions script contains the following lines, which runs all our Go unit tests:
+```yaml
+# Run go unit tests
+- name: Unit testing
+run: |
+    go test -v ./...
+# Run simulator integration tests
+- name: Simulator tests
+run: |
+    pip install -r ./tools/requirements.txt
+    pytest ./tools/test_sim_compliance.py
+```
+
+## Enhance your CI Pipelines with at least three static analysis tools
+In our CI/CD setup we have included three static analysis tools. These tools help ensure high software quality and maintainability. The three static analysis tools we have implemented are:
+
+[Lint](https://github.com/golang/lint) for Go - helps flagging programming errors, bugs and stylistic errors. Runs from the following action:
+```yaml
+#Run linter to check if new code follows the style guidelines
+- name: lint
+run: if [ "$(gofmt -s -l . | wc -l)" -gt 0 ]; then exit 1; fi
+```
+
+[CodeQL](https://codeql.github.com/) - for flagging vulnerabilities across the codebase. Defined in [codeql.yml](https://github.com/Lindharden/DevOps/blob/main/.github/workflows/codeql.yml).
+
+[Snyk](https://snyk.io/) - for finding and fixing vulnerabilities in our dependencies. Defined in [snyk-security.yml](https://github.com/Lindharden/DevOps/blob/main/.github/workflows/snyk-security.yml).
+
+## Add Maintainability and Technical Debt estimation tools to your projects
+We added [Sonarqube](https://www.sonarsource.com/products/sonarcloud/) and [Code Climate](https://codeclimate.com/) to our application. These tools will create comments on our pull-requests, detailing any potential problems with the committed code. Additionally we can visit the websites to view detailed statistics and improvements which are proposed to our code.
+
+## Mapping subsystems
+The following UML diagram maps all the subsystems of our Minitwit application.
+![Minitwit UML](img/minitwit_uml.png)