[Event Hubs Client] Stabilize Live Tests #5451
Assignees
Labels
Milestone
Comments
jsquire
added
Client
jsquire
changed the title
jsquire
added a commit
to jsquire/azure-sdk-for-net
that referenced
this issue
Apr 24, 2019
Project Structure
- Solution and projects adapated to central engineering system structure,
format, and conventions
- Target platforms updated to conform to the central engineering system. For
this initial effort, the Xamarin.iOS10 target has been converted to
netstandard1.4, which is compatible with the previously configured iOS verison.
- Target platforms have been updated to only include uap10.0 on Windows and when a flag is passed
to indicate that MsBuild is being used, as UWP currently cannot be build from .NET core. In the
current configuration for the engineering system, this means that the uap10.0 target will not be
built; discussions for adding support have begun.
NOTE: This is subject to change pending the outcome of discussions with the engineering team
on the best way to enable support within the build system.
- Updated conditional compilation to recognize netstandard1.4 and uap10.0 as the target platforms
for consumers which the Identity Model for Active Directory package does not support
client certificate use; the constructs previously hidden for iOS and UWP are now
hidden for netstandard1.4 and UWP.
- Forced a target platform of x64 for Service Fabric Processor and Test projects,
due to the restrictions imposed by the Service Fabric library references.
- Configured overrides to the engineering system default assembly signing key, to
allow for use of the existing Azure Messaging key while the client library is in Track 1,
to avoid breaking changes.
- Normalized and updated package references to known-good versions used by the centralized
SDKs and attempted to remove fragmentation between target platform references.
Azure Resource Templates
- Converted values which are assumed by the test infrastructure from parameters to variables
with the expected names. This ensures that deployment is compatible with the test suite without
a user needing to have awareness of the expected names.
ReadMe
- Restructured and updated content to snap to the Azure SDK repository ReadMe template, as closely
as possible for a Track 1 migration.
Event Hubs Tests
- Designated tests that interact with Azure services as "Live" tests, allowing them to be filtered when
Azure environments are not available or when it is desirable to run only fast, self-contained tests, such
as build verification for pull requests.
- Moved hardcoded values tied to Azure resources into the TestConstants file, so that well-known
and assumed values are centrally located.
- Updated the approach for reading environment variables to a lazy approach instead of in a static
constructor, allowing tests that do not use Azure resoruces to make use of TestUtility functionality
without the need for a set of "Live" environment variables.
- Moved test utility and infrastructure classes to a dedcated folder for better organization and in
anticipation of additional test infrastructure additions during the upcoming test stabilization.
- Added configuration for the XUnit test runner to enable diagnostics output and in anticipation of
parallelization of the tests during stabilization work.
- Added access modifiers to tests in order to satisfy code analysis rules for unused private members.
- Small fixes to async patterns in some tests, to await rather than returning the task directly, per
XUnit recommended practices.
- Small additions to failure messages, to grant additional context and highlight actual vs expected values.
- Temporarily set some unstable tests in the the ServiceFabricProcessor area to be skipped
until for the initial migration; to be investiated and stablized as part of issue Azure#5451.
jsquire
added a commit
that referenced
this issue
Apr 24, 2019
* [Event Hubs Client] Code Drop
Initial drop of the Event Hubs client library code from its stand-alone repository,
based on commit d8c1e3ef321f3873b311e3b795c5e19d78e1de29, by Serkant Karaca on
March 28, 2019
* [Event Hubs Client] Initial Migration
Project Structure
- Solution and projects adapated to central engineering system structure,
format, and conventions
- Target platforms updated to conform to the central engineering system. For
this initial effort, the Xamarin.iOS10 target has been converted to
netstandard1.4, which is compatible with the previously configured iOS verison.
- Target platforms have been updated to only include uap10.0 on Windows and when a flag is passed
to indicate that MsBuild is being used, as UWP currently cannot be build from .NET core. In the
current configuration for the engineering system, this means that the uap10.0 target will not be
built; discussions for adding support have begun.
NOTE: This is subject to change pending the outcome of discussions with the engineering team
on the best way to enable support within the build system.
- Updated conditional compilation to recognize netstandard1.4 and uap10.0 as the target platforms
for consumers which the Identity Model for Active Directory package does not support
client certificate use; the constructs previously hidden for iOS and UWP are now
hidden for netstandard1.4 and UWP.
- Forced a target platform of x64 for Service Fabric Processor and Test projects,
due to the restrictions imposed by the Service Fabric library references.
- Configured overrides to the engineering system default assembly signing key, to
allow for use of the existing Azure Messaging key while the client library is in Track 1,
to avoid breaking changes.
- Normalized and updated package references to known-good versions used by the centralized
SDKs and attempted to remove fragmentation between target platform references.
Azure Resource Templates
- Converted values which are assumed by the test infrastructure from parameters to variables
with the expected names. This ensures that deployment is compatible with the test suite without
a user needing to have awareness of the expected names.
ReadMe
- Restructured and updated content to snap to the Azure SDK repository ReadMe template, as closely
as possible for a Track 1 migration.
Event Hubs Tests
- Designated tests that interact with Azure services as "Live" tests, allowing them to be filtered when
Azure environments are not available or when it is desirable to run only fast, self-contained tests, such
as build verification for pull requests.
- Moved hardcoded values tied to Azure resources into the TestConstants file, so that well-known
and assumed values are centrally located.
- Updated the approach for reading environment variables to a lazy approach instead of in a static
constructor, allowing tests that do not use Azure resoruces to make use of TestUtility functionality
without the need for a set of "Live" environment variables.
- Moved test utility and infrastructure classes to a dedcated folder for better organization and in
anticipation of additional test infrastructure additions during the upcoming test stabilization.
- Added configuration for the XUnit test runner to enable diagnostics output and in anticipation of
parallelization of the tests during stabilization work.
- Added access modifiers to tests in order to satisfy code analysis rules for unused private members.
- Small fixes to async patterns in some tests, to await rather than returning the task directly, per
XUnit recommended practices.
- Small additions to failure messages, to grant additional context and highlight actual vs expected values.
- Temporarily set some unstable tests in the the ServiceFabricProcessor area to be skipped
until for the initial migration; to be investiated and stablized as part of issue #5451.
This was referenced Apr 24, 2019
Merged
mentat9
pushed a commit
to mentat9/azure-sdk-for-net
that referenced
this issue
Jun 10, 2019
* [Event Hubs Client] Code Drop
Initial drop of the Event Hubs client library code from its stand-alone repository,
based on commit d8c1e3ef321f3873b311e3b795c5e19d78e1de29, by Serkant Karaca on
March 28, 2019
* [Event Hubs Client] Initial Migration
Project Structure
- Solution and projects adapated to central engineering system structure,
format, and conventions
- Target platforms updated to conform to the central engineering system. For
this initial effort, the Xamarin.iOS10 target has been converted to
netstandard1.4, which is compatible with the previously configured iOS verison.
- Target platforms have been updated to only include uap10.0 on Windows and when a flag is passed
to indicate that MsBuild is being used, as UWP currently cannot be build from .NET core. In the
current configuration for the engineering system, this means that the uap10.0 target will not be
built; discussions for adding support have begun.
NOTE: This is subject to change pending the outcome of discussions with the engineering team
on the best way to enable support within the build system.
- Updated conditional compilation to recognize netstandard1.4 and uap10.0 as the target platforms
for consumers which the Identity Model for Active Directory package does not support
client certificate use; the constructs previously hidden for iOS and UWP are now
hidden for netstandard1.4 and UWP.
- Forced a target platform of x64 for Service Fabric Processor and Test projects,
due to the restrictions imposed by the Service Fabric library references.
- Configured overrides to the engineering system default assembly signing key, to
allow for use of the existing Azure Messaging key while the client library is in Track 1,
to avoid breaking changes.
- Normalized and updated package references to known-good versions used by the centralized
SDKs and attempted to remove fragmentation between target platform references.
Azure Resource Templates
- Converted values which are assumed by the test infrastructure from parameters to variables
with the expected names. This ensures that deployment is compatible with the test suite without
a user needing to have awareness of the expected names.
ReadMe
- Restructured and updated content to snap to the Azure SDK repository ReadMe template, as closely
as possible for a Track 1 migration.
Event Hubs Tests
- Designated tests that interact with Azure services as "Live" tests, allowing them to be filtered when
Azure environments are not available or when it is desirable to run only fast, self-contained tests, such
as build verification for pull requests.
- Moved hardcoded values tied to Azure resources into the TestConstants file, so that well-known
and assumed values are centrally located.
- Updated the approach for reading environment variables to a lazy approach instead of in a static
constructor, allowing tests that do not use Azure resoruces to make use of TestUtility functionality
without the need for a set of "Live" environment variables.
- Moved test utility and infrastructure classes to a dedcated folder for better organization and in
anticipation of additional test infrastructure additions during the upcoming test stabilization.
- Added configuration for the XUnit test runner to enable diagnostics output and in anticipation of
parallelization of the tests during stabilization work.
- Added access modifiers to tests in order to satisfy code analysis rules for unused private members.
- Small fixes to async patterns in some tests, to await rather than returning the task directly, per
XUnit recommended practices.
- Small additions to failure messages, to grant additional context and highlight actual vs expected values.
- Temporarily set some unstable tests in the the ServiceFabricProcessor area to be skipped
until for the initial migration; to be investiated and stablized as part of issue Azure#5451.
|
For ServiceFabricProcessor, most of these were fixed by #7446. EventException test has a separate issue with the error count being off by one. |
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
The tests adopted from the standalone repository, which were designated as Live are not fully self-contained nor resilient, which has resulted in some unpredictable failures when run as part of the nightly runs.
In order to ensure that the nightly run is reliable, the Event Hubs results should be investigated and tests that are not reliable should be analyzed.
If the test is not current and is not needed to verify functionality of the service, it should be deleted.
If the test can be improved in a reasonable amount of time and effort, it should be.
If the test is needed and requires more than a reasonable effort, it should be flagged to be skipped and a separate issue opened to cover the work required to fix the test.
The text was updated successfully, but these errors were encountered: