E2E failing links test is flaky

[matzf]

The E2E failing links test has been flaky since #4168.
The general problem seems to be that the await-connectivity script waits for a path to be established between any two ASes, but the "E2E failing links" test requires specific paths to have been discovered before continuing to disable routers.
Consequently, possible fixes could be:

• wait for a fixed amount of time
• extend the await-connectivity script to wait for specific path segments
• adapt the await-connectivity script to wait for all paths to be established, e.g. by waiting until there are no new segments after one full beaconing period.

[matzf]

Update: I did some investigation (#4356), suspecting that the problem mainly lies with the error and timeout handling end-to-end test utility. However, while I found that these can be improved, this is not the (main) issue here.

The problem is related to the caching of path segments in the control service; once path segments are queried, the result is kept in the cache for a relatively long time (path.query_interval, defaults to 5 minutes). The "Refresh" flag of the sciond path requests does not propagate to the path segment requests in the control service.
If the "surviving" segments of the failing link scenario have not been registered before the first path segment request is made, the test will fail.
This suggests a possible alternative, going beyond just fixing the test; rethink the path query caching strategy, to better take into account the interaction between the different caching levels and the usage requirements from SCION applications.