📁 Develop a SFTP server on AWS AP to transfer data from Essex Police to MoJ for BOLD Programme

[jacobwoffenden]

Transferred from https://github.com/ministryofjustice/analytics-platform-infrastructure/issues/1098

User Story

As a Essex Police Data Engineer I need for a SFTP server to be setup on AWS AP so that I can send data to the MoJ to allow for data transfer as agreed with MoJ as part of the BOLD Programme.

As a MoJ Data Engineer I need for the data to land into a S3 bucket on the AWS AP so that I can transform it into usable data for analysts on the AP, who will use the data for the BOLD Programme.

User Type(s)

Might need help understanding what to put here.

Value

Would allow for data to be transferred from Essex police to MoJ for the BOLD Programme. Without setting up the SFTP server the data cannot be transferred as Essex Police are unable to entertain another approach.

The value of the ask would be to allow for data transfer to occur for a BOLD Programme, which the MoJ is committed to until March 2024.

Questions / Assumptions / Hypothesis

If we develop an STFP server on AWS and provide an end-point and auth credentials to Essex Police this will allow them to use already established technologies at their end to transfer data in an automated and secure manner to the MoJ.

Definition of done

• S3 bucket has been created where data can land and is accessible by MoJ Data Engineers
• SFTP server has been created which allows for Essex police to securely transfer data to MoJ
• SFTP server allows for daily data transfers for Essex Police
• Log and event triggers inform people of when data has been uploaded (or not) so we can track if Essex Police are uploading data as expected

Reference

How to write good user stories

[jacobwoffenden]

Deploying AWS Transfer Service may solve this too https://github.com/ministryofjustice/data-platform-support/issues/174

[mshodge]

Update to requirements on this one: the method now needs to allow us to send data to others, not just receive data.

[mshodge]

Proposed (simple) framework, for review:

[jhpyke]

Refinement (24/10/23): Product team to be asked to contact BOLD to ensure this is still a relevant ask, and if so ticket needs updating/closing.

[YvanMOJdigital]

@Ed-Bajo needs a status update

[Gary-H9]

@Ed-Bajo speaking to Bold this week.

[julialawrence]

Edwin to chase up need.

[Ed-Bajo]

This ticket is on hold and Edwin to contact the BOLD team for the way forward.

[mshodge]

Hi @Ed-Bajo we have just got some more movement from Essex Police, and more questions I am afraid. One being, what will the IP address be that we will send them data via the SFTP. If someone could reach out to me to discuss/let me know, that would be great. Thanks.