E2E tests runner

[josecelano]

A new binary to run E2E tests:

• Build the docker image.
• Run the docker image.
• Wait until the container is healthy.
• Parse logs to get running services.
• Build config file for the tracker_checker.
• Run the tracker_checker.
• Stop the container.

[codecov]

Codecov Report

Attention: 195 lines in your changes are missing coverage. Please review.

Comparison is base (dee86be) 77.61% compared to head (ec13fb4) 75.99%.

Files	Patch %	Lines
src/e2e/runner.rs	0.00%	103 Missing ⚠️
src/e2e/docker.rs	0.00%	75 Missing ⚠️
src/e2e/temp_dir.rs	0.00%	13 Missing ⚠️
src/e2e/logs_parser.rs	91.11%	4 Missing ⚠️

Additional details and impacted files

@@             Coverage Diff             @@
##           develop     #643      +/-   ##
===========================================
- Coverage    77.61%   75.99%   -1.62%     
===========================================
  Files          131      135       +4     
  Lines         8555     8791     +236     
===========================================
+ Hits          6640     6681      +41     
- Misses        1915     2110     +195     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[josecelano]

Hi @da2ce7 I'm also experimenting with using Rust instead of Bash for the CI scripting. We have discussed in the past that it would be nice to write those scripts in Rust too. I think in this case it's a perfect fit.

[josecelano]

Hi @da2ce7 I'm using Docker for the E2E test environment like in the Index. In the Index we needed because we wanted to also run the Tracker. Here we don't need it, we could simply run cargo run but I'm using it to test also containerization. This way E2E tests will cover everything. The problem is the container build is made again in the testing workflow and it takes 10 minutes.

Another problem with Docker is, we can not use port 0 to assign a free port becuase we need to publish the port when we run docker.

So, I'm considering not using Docker. What do you think?

[josecelano]

Hi @da2ce7, some comments:

1. Independent crate for each CI script?

Maybe this script should be an independent crate in the contrib/dev-tools dir.

It's like this script in Bash but in Rust.

Rigth now, there is no coupling between the main crate and this binary. And I don't like having CI/Testing code in the production package.

2. Use docker?

See #643 (comment)

3. Use test-helpers package to build the configuration for the tracker?

I'm using a file in share/default/config. We could use test-helpers or configuration package. However I prefer to do it similar to what I would do it manually to test the tracker.

4. Check that there is no panic in the logs

I could add one more test. I could check that there are no "panics" in the logs. That was one of the reasons I started adding E2E tests. There were some panics in the bootstrapping of the services that made the halt signal not work. But the servers were running anyway, so these panics were not detected by tests.

5. Use an env var to pass conf to the tracker_checker

That way we don't even need the temp dir. We could also remove the file at the end and do all in the root folder.

[josecelano]

ACK ec13fb4

[josecelano]

With an independent crate for CI scripts, we need to run their tests in new workflows. Maybe that's too complex. We could use one single crate for all the scripts with one binary per script.