Merge Dev into Master #4940
Merged
Merge Dev into Master #4940
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
EES-5158 update national stats branding
EES-5175 and EES-5177 data catalogue updates
…or Function App and access token retrieval in the Admin App Service.
…naged-identity-security-to-data-processor-2 EES-5119 - adding Managed Identity authentication to the Data Processor
…ion into Bicep for configuring Health Checks and creating metrics. Reduced retry attempts on deploying Data Processor code and removed manual sleep, as sleeps are included in retry mechanism.
…single build with rotating password policy to enable stability in various components of the service using PSQL. Various responses to PR comments.
…for core storage and content db connections
…ta-processor-function-app-health-checks Ees 5147 add data processor function app health checks
EES-5119 - slight fix for pluralised filename
* chore(deps): update minor backend dependencies * Knock xunit back down to version 2.7.1
…t-parallel-infrastructure-runs EES-5119 - prevent parallel runs of API infrastructure pipeline
…public-data-db-connection-on-dev EES-XXXX - enabling Public Data DB connection on Dev for Admin and Publisher
EES-5159 hook up data set preview, variables and footnotes sections
EES-5187 Remove compression of CSV metadata file in `CopyCsvFiles` function
…-exclusive-lock-option EES-5119 - removing "batch" option in favour of exclusive lock setup
…ccess to Core Storage
…blic-api-subnets-access-to-core-storage EES-5147 - giving the Public API Data Processor Function App subnet access to Core Storage
EES-5158 increase stats logo size
…uilder Signed-off-by: Nicholas Tsim <nicholastsim@gmail.com>
This simplifies the `DataSetPublishing.PublishDataSets` implementation, removes the unnecessarily verbose comments and simplifies naming across the board.
Signed-off-by: Nicholas Tsim <nicholastsim@gmail.com>
This removes the `Add*TestData` and `Verify*DbContext` methods that we currently have in the various `*DbUtils` files. Instead of these methods, we now adopt a pattern that is consistent with the public API integration tests using `AddTestData` and `GetDbContext`. Signed-off-by: Nicholas Tsim <nicholastsim@gmail.com>
…ion for Admin. Leaving Publisher connection as it works for Functions.
…-connection-string-to-custom EES-XXXX - changing connection string type to Custom for PSQL connection for Admin
EES-5185 Various cleanup for creating and fetching data sets in admin
…st-snapshots chore(tests): Update test snapshots 06-06-2024
EES-5163 Clean up DataSetFileMeta and remove migration endpoint
…iles to avoid long file paths
EES-5188 update node to v20.14.0
EES-5197 Use temp directory rather than assembly directory for Public API data set version files
…4902) * EES-5172 Processor - [WIP] Adding Delete endpoint to the Processor - Proof of concept of distributed transactions * EES-5172 Processor - Fixing bug where our custom retry execution policy was erroring due to an ambient transaction In order to be able to use retry execution strategies alongside an ambient transaction, we have to be explicit about which unit of work the execution strategy is executing on. See https://learn.microsoft.com/en-us/ef/core/miscellaneous/connection-resiliency#execution-strategies-and-transactions * EES-5172 Processor - Fetching all data up-front before deleting any data to avoid partial transactions being committed due to an exception not being thrown when an entity cannot be found * EES-5172 Processor - HttpTrigger Now returns `NotFoundResult` when this is returned from the service layer * EES-5172 Processor - Now throws a `KeyNotFoundException` when the `ReleaseFile` associated with a `DataSetVersion` cannot be found * EES-5172 Admin - Adding new endpoint to delete a `Draft` `DataSetVersion` and it's associated data * EES-5172 Admin - Wrapping the trigger in a try/catch to prevent hanging when an exception is thrown This is a temporary fix. The cause of this bug is unknown. The bug causes the request to hang until a timeout occurs when an exception is thrown. There is an issue here (Azure/azure-functions-dotnet-worker#2476) which we are tracking. If the problem gets fixed, then we will unwrap the trigger. * EES-5172 Processor - Guarding against deleting a `Draft` DataSetVersion * EES-5172 Processor - Adding integration tests for the DELETE endpoint * EES-5172 Admin - Adding integration tests * EES-5172 Admin - Adding a test to cover the case where the `ProcessorClient` throws a `HttpRequestException` * EES-5172 Changing the criteria for if a data set version can be deleted A data set version can now be deleted if it is in any of the following statuses: - `Failed` - `Mapping` - `Draft` - `Cancelled` * EES-5172 Undoing unnecessary change to Processor.Tests.csproj file * EES-5172 Addressing PR comments * EES-5172 Addressing rebase issues * EES-5172 Removing empty line * EES-5172 Fixing tests * EES-5172 Renaming request parameter and constraining the path to use a Guid
mmyoungman
approved these changes
Jun 7, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR merges Dev branch into M<aster for upcoming deploys to Pre-Prod and Prod environments.