Please help add regression test runs for identity

[xiangyan99]

Now we have regression runs for azure-core.

Can we add the regression tests for identity as well?

What we want is to install latest bits and run tests against last GA release. So we can make sure we don't introduce breaking changes between two GA releases.

[scbedd]

• azure-identity is checked against the latest azure-core and earliest azure-core in it's mindependency stage.
• azure-core regression checks find the earliest and latest released version of azure-identity that match current azure-core and runs tests against them.

I think these are covered scenarios. Closing. Feel free to reopen if you want @xiangyan99.

[xiangyan99]

@scbedd what we are looking for is regression validation.

e.g. currently we install latest GAed identity and run storage tests.

What we want to validate is install the live identity from main and run those tests to catch breaking changes/regressions.

Do you know if we have such coverage in our current matrix?

Thanks.

[scbedd]

What we want to validate is install the live identity from main and run those tests to catch breaking changes/regressions.

I laid out the two ways we test, can you give me a bit more detail about what you mean by regression?

What we want to validate is install the live identity from main and run those tests to catch breaking changes/regressions.

Doesn't the mindependency check do literally exactly this? Current identity from main (or your pr) running against earliest and latest azure-core that it can depend on?

[scbedd]

Had a chat with @xiangyan99 about this.

We need a new regression step that runs based on dev_requirements contents + actual dependencies.

This is a live test invocation, it has nothing to do with recorded tests.

One way we could recognize this is to check dev_requirements for any azure-* dependency.
if present, that is a package that should be regression tested.
A regression test means running live tests for that target using the identity from disk NOT PYPI

[github-actions]

Hi @xiangyan99, we deeply appreciate your input into this project. Regrettably, this issue has remained inactive for over 2 years, leading us to the decision to close it. We've implemented this policy to maintain the relevance of our issue queue and facilitate easier navigation for new contributors. If you still believe this topic requires attention, please feel free to create a new issue, referencing this one. Thank you for your understanding and ongoing support.