Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Pivotal ID # 184975504: S3 submission resubmission #702

Merged

Conversation

Juan-EBI
Copy link
Contributor

@Juan-EBI Juan-EBI commented Apr 25, 2023

https://www.pivotaltracker.com/story/show/184975504

  • added support for toxic risk validation
  • added support for s3 Fire transfer into NFS

Copy link
Contributor

@jhoanmanuelms jhoanmanuelms left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Please consider my comments before merging

assertThat(webClient.submitSingle(submission, TSV, FIRE)).isSuccessful()
val fireSub = submissionRepository.getExtByAccNo("S-STR-MODE-4", includeFileListFiles = true)

// We upload files manually into S3 to simulate fire S3 support.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just to clarify, is this to validate the case where the imaging team uploads the files directly to FIRE? If so, I don't think this is the right place to validate this logic since the idea of the transfer function is for submissions that already exist and will be transferred as-is so it seems weird to me we're adding files afterward.

I think the proper test case for this would be here like this:

  1. Submit to FIRE
  2. Upload extra files manually
  3. Resubmit to NFS including the manually uploaded files in the new version

Copy link
Contributor Author

@Juan-EBI Juan-EBI Apr 28, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, this is not because of that. This changes are because we need to simulate fire mechanism that make every file uploaded through http it make it avaiable in s3.

Copy link
Contributor Author

@Juan-EBI Juan-EBI Apr 28, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

wondering if the comment need to be better to clarify this


internal const val EXCEL_FILE_REQUIRED = "Excel file is required for Eu-ToxRisk submissions"
internal const val SKIP_VALIDATION_ATTR = "QMRF-ID"

class EuToxRiskValidator(
private val restTemplate: RestTemplate,
private val validationProperties: ValidatorProperties
private val validationProperties: ValidatorProperties,
private val fireClient: FireClient,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

@@ -25,11 +26,6 @@ internal class RetryWebClient(
template.execute(opt) { fireClient.unsetPath(fireOid) }
}

override fun downloadByPath(path: String): File? {
val opt = "Download file path='$path'"
return template.execute(opt) { fireClient.downloadByPath(path) }
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since we're removing this, I think we should also remove the http client method and the related tests since it won't be required anymore

import java.nio.file.Files
import java.nio.file.StandardCopyOption

class S3Client(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we're missing unit tests for this client

@Juan-EBI Juan-EBI merged commit fab23a5 into master May 2, 2023
@Juan-EBI Juan-EBI deleted the feature/pivotal-184975504-S3-submission-resubmission branch May 2, 2023 13:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants