Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Pivotal ID # 182800115: Folder Submissions handling #590

Merged

Conversation

Juan-EBI
Copy link
Contributor

@Juan-EBI Juan-EBI commented Jul 24, 2022

https://www.pivotaltracker.com/story/show/182800115

  • handled 404 when not files found by md5.
  • added zip file handling logic.

@@ -63,17 +64,12 @@ internal class FireWebClient(
}

override fun findByMd5(md5: String): List<FireApiFile> =
template.getForObject<Array<FireApiFile>>("/objects/md5/$md5").toList()
template.getForObjectOrNull<Array<FireApiFile>>("/objects/md5/$md5").orEmpty().toList()
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here we're actually calling nonexisting endpoints which are giving us the 404. For example, if the md5 is empty we would hit /objects/md5/ which doesn't exist. Maybe we should not call the endpoints when the values are empty and just return an empty list instead of handling the 404 for a call we know will fail? 🤔

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

error on terminal for me , was md5 not found.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, I thought it was coming from the standard http error but apparently they decided this design but, at least, I think we could save one request since we know for sure empty md5 or path will not return any result don't you think?

@@ -25,6 +25,7 @@ dependencies {
api(project(SubmissionPersistenceCommonApi))

implementation(Arrow)
implementation("commons-codec:commons-codec:1.15")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should add a dependency in Dependencies.kt

@@ -29,21 +33,47 @@ class FireService(
* different path) and if so, even if the file exists in FIRE, it gets duplicated to ensure consistency.
*/
fun getOrPersist(sub: ExtSubmission, file: ExtFile): FirePersistResult {
val expectedPath = "/${sub.relPath}/${file.relPath}"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I had made this change because the paths are the same in both places which I think makes clear that no matter where the file is processed, we use the same path

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes the problem is that this is not the path when file is a folder. As we need to add the .zip that is the reason I move the call after compressing the file and creating extDirectory.

I did update the code to have single declaration please take a look again.

return target
}

fun calculateMd5(dir: File): String {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since we're always compressing the directory before uploading it to fire, why do we need to calculate the md5 in this way? Or is this the same value as the compressed directory?

I'm just asking because I'm not sure fire will take this md5 for the upload as valid and we'll need to recalculate it anyway and also, if we're trying to find it by md5 in order to reuse it, I'm not sure if this md5 will do


object ZipUtil {

fun pack(sourceDir: File, zipFile: File) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

Copy link
Contributor

@jhoanmanuelms jhoanmanuelms left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@jhoanmanuelms jhoanmanuelms merged commit 0cd9ee8 into master Jul 26, 2022
@jhoanmanuelms jhoanmanuelms deleted the feature/pivotal-182800115-Folder-submissions-handling branch July 26, 2022 13:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants