new cat for readiness health check feature - linux & windows

[ameowlia]

Are you submitting this PR against the develop branch?

✅ Yes.

What is this change about?

This PR adds a new CAT for readiness health checks, a feature that is coming out with new versions of CAPI and Diego soon.

Please provide contextual information.

• CFF RFC for adding readiness health checks
• PR to add this to Cloud Controller

What version of cf-deployment have you run this cf-acceptance-test change against?

⛔️⛔️⛔️TBD⛔️⛔️⛔️

Please check all that apply for this PR:

• introduces a new test --- Are you sure everyone should be running this test?
• changes an existing test
• requires an update to a CATs integration-config

Did you update the README as appropriate for this change?

• YES
• N/A

If you are introducing a new acceptance test, what is your rationale for including it CATs rather than your own acceptance test suite?

We are adding a major new feature to CF. This CAT will test one basic workflow for that feature.

How many more (or fewer) seconds of runtime will this change introduce to CATs?

220-ish seconds. Sorry :(

What is the level of urgency for publishing this change?

• Urgent - unblocks current or future work
• Slightly Less than Urgent

Tag your pair, your PM, and/or team!

• @mariash

[ctlong]

I've left a bunch of comments for your review. Generally it looks good.

🤔 I do wonder whether there's a way we can split up the tests to reduce the time this adds to CATs. I'm thinking that maybe the test could be split in two:

1. Push an app that will fail the readiness health check (i.e. push with --no-start and set an env var that will cause readiness to health check to fail at first), then reactivate it with the proxy app c2c and see everything work.
2. Push an app that will pass the readiness health check, then deactivate the readiness health check with the endpoint, and see it removed from the routing chain.

[geofffranks]

Feedback has been addressed/commented on. I looked into splitting them out into multiple tests to parallelize. Unfortunately we don't have any feedback from an app starting up that it has failed the initial readiness checks, other than absence of a message. Short of waiting for a while, we can't differentiate "readiness hasn't checked yet" and "readiness ran and failed." This makes the second test run slightly longer than when a single test runs both cases.

[ctlong]

I looked into splitting them out into multiple tests to parallelize... This makes the second test run slightly longer than when a single test runs both cases.

@geofffranks and I chatted about this and discovered that he accidentally ran his comparison with no parallelization, as ./bin/test does not run CATs in parallel by default. The --procs flag needs to be explicitly provided.

👉 I'm waiting for that comparison to be re-ran with parallelization before I review this PR again. 👈

[geofffranks]

I re-ran with --procs 8 and as a single test, it executed in 104s. As a split test, it took 148s.

The main difference is that when split, the un-ready app has to wait in a Consistently() for 25s to ensure the app never becomes ready, which the original test doesn't have.

Going to leave this as a single test.

[ctlong]

I'm still seeing the local path C:\Users\gfranks\source\repos\cf-acceptance-tests referenced throughout the changes to Nora. I don't know .NET well enough to know when it's avoidable or not, but hopefully most are 🤞

Can you all please search for the term gfranks in the assets/nora path (make sure to check hidden and ignored files!) and remove them wherever possible.

---

Edit:

$ rg -i --hidden --no-ignore --files-with-matches gfranks
NoraPublished/Nora.csproj.user
NoraPublished/Properties/PublishProfiles/NoraPublished.pubxml.user
NoraPublished/Properties/PublishProfiles/FolderProfile.pubxml
NoraPublished/Properties/PublishProfiles/FolderProfile.pubxml.user
Nora/Nora.csproj.user
Nora/Properties/PublishProfiles/FolderProfile.pubxml.user
Nora/Properties/PublishProfiles/FolderProfile.pubxml
Nora.Tests/obj/Release/Nora.Tests.csproj.FileListAbsolute.txt

[ctlong]

Aaaah shoot, can't believe I didn't catch this earlier 🤦

The base branch for this PR needs to be develop. Can you please change the base branch and rebase.

PR is against the wrong branch

There are merge conflicts