Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Data validation tool spikes #22

Merged
merged 3 commits into from
Oct 24, 2023
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
52 changes: 52 additions & 0 deletions source/decision-record.html.md.erb
Original file line number Diff line number Diff line change
Expand Up @@ -528,3 +528,55 @@ source url that is compatible with vector-based map libraries.
- The tile server simply serves the tile requested - there is no expensive logic or spatial queries required.
- The generation of the tilesets is an added time cost to our CI/CD pipeline.
- We should be careful not to use conversion options that change the underlying data (e.g. removing polygons/points)

## 18. Use NodeJS to serve the frontend of the new data validation tool and interact with other APIs

Date: 2023-10-24

#### Status

Pending

#### Context

We need some way to serve our frontend to users, and handle forwarding requests onto our other APIs needed for validation. at the moment the requirements on this are fairly limited, but we want the room to grow if needed.

#### Decision

We will start with a simple nodeJS application and will review if this was the right call a couple of weeks from now.
All the buisness logic should be delegated to other APIs, and exception to this will be documented.

#### Consequences

- This approach would have slight performance improvements over fastAPI.
- Prototypes produced by our designer Adam Silver can be easily used as a starting point as they are built with nunjucks and the gds.
- Frontend developers may find it easier to onboard as they will be familiar with the tech stack. and wont need to upskill in python.
- We should be careful not to include too much business logic within the nodeJS application, or risk needing our developers to upskill in nodejs and fastapi.
- We will be able to use the official GDS frontend toolkit, which will make it easier to build a consistent UI. as opposed to using the unofficial python port / nunjucks implementation of the GDS frontend toolkit.


## 19. Use PyTest and JSONSchema as the Contract validation and testing solution
Date: 2023-10-24
#### Status
Pending
#### Context
Ideally, we would like to:
- test messages flowing in and out of our API endpoints - Requests & Responses
- establish message schemas for the service-oriented elements of the Products we create
#### Decision
We will develop a suite of tests (over time) that will typically
be run by the CI/CD pipeline.

The tests will be based on the PyTest framework as it has proven to
be an effective testing framework, and is already used extensively in our code base.

The tests will focus on:
- validating that correct messages produce valid,
and expected responses.
- validating that incorrect messages produce a suitable error response
#### Consequences
- The approach assumes that the API front end is capable of injecting the validation service as a dependency.
- The approach may prove too simplistic and need replacing with something better suited to the purpose.
- The first tests will be based on the FastAPI framework. If we later decide to use another framework,
the tests will extensive re-factoring to continue working