Pivotal ID # 179784398: Improve all in one itest to include file list #450

Juan-EBI · 2021-10-06T11:15:57Z

https://www.pivotaltracker.com/story/show/179784398

…studies-backend-services into feature/pivotal-179784398-Improve-all-in-one-itest-to-include-file-list

jhoanmanuelms

LGTM. Just a couple of thoughts.

jhoanmanuelms · 2021-10-07T18:08:27Z

commons/commons-test/src/main/kotlin/ebi/ac/uk/test/TemporaryFolderExt.kt

+/**
+ * Creates a file with the given name and content or replaces it if already exist.
+ */
+fun TemporaryFolder.createOrReplaceFile(fileName: String, content: String): File {


Couldn't this lead to failures? Maybe it'd be better just cleaning the temp folder after each test if required.

yes totally agree

jhoanmanuelms · 2021-10-07T18:10:38Z

...ission-webapp/src/itest/kotlin/ac/uk/ebi/biostd/itest/assertions/AllInOneSubmissionHelper.kt

    }

-    private fun assertSubmissionFiles(submission: ExtSubmission) {
+    private fun assertSubmissionFiles(submission: ExtSubmission, checkTabFiles: Boolean) {


As a wise friend once told me, boolean flags in methods aren't usually a good idea. Maybe it's better to have separated methods or just a method for specifically checking the page tab files and call it whenever necessary.

Also, I'd suggest checkPageTabFiles to keep consistency.

yes this is temporally as we do not support pagetabe files in mysql. Changing it still.

jhoanmanuelms · 2021-10-07T18:17:00Z

.../kotlin/ac/uk/ebi/biostd/itest/test/submission/submit/AllInOneMultipartFileSubmissionTest.kt

+        fun `submit all in one multipart TSV submission`() {
+            val (submission, fileList, files) = submissionSpecTsv(tempFolder, "S-EPMC124")
+            webClient.uploadFile(fileList)
+            files.forEach { webClient.uploadFile(it.file, it.folder) }


Nice, this is something we were missing

Feature Pivotal ID # 177839764: FIRE FileProcessingService (#415) Pivotal ID # 178930818: Extended File List Endpoint (#418) Pivotal ID # 179363552: Add fire files interface source support (#436) Pivotal ID # 178461698: FIRE Database Persistence (#437) Pivotal ID # 179389139: FIRE Directories Persistence (#439) Pivotal ID # 179361117: Add support for groups administration (#434) Pivotal ID # 179609612: Submit Large Files To Fire (#444) Pivotal ID # 179418482: Persist PageTab FIRE (#440) Pivotal ID # 179513930: Add Warning For Empty File Lists (#448) Pivotal ID # 179040289: Async Migration (#451) Pivotal ID # 179810581: Files files storage (#456) Pivotal ID # 180013760: Add submissions schema version field (#463) Pivotal ID # 179922324: File Mode CLI Option (#465) Pivotal ID # 180152694: File List Validation (#473) Pivotal ID # 179921481: Fire ftp publishing (#467) Pivotal ID # 180253535: Submission storage mode (#468) Pivotal ID # 180282390: Reset password should activate user (#470) Bugfix Pivotal ID # 179145538: Handle corrupted files in PMC (#429) Pivotal ID # 178331250: Files Names Encoding (#431) Pivotal ID # 179435581: Deleting Submissions Isn't Working (#442) Pivotal ID # 179904973: Fix Notification URL (#455) Pivotal ID # 179922291: Inner File List Referenced Files (#457) Pivotal ID # 179867189: TSV Quotes Deserialization (#462) Pivotal ID # 180345449: handle empty title submission email notification (#472) Chore Pivotal ID # 179055347: Update AE user activation email (#420) Pivotal ID # 179105055: Update Java Home (#421) Pivotal ID # 179144369: Add tsv library (#426) Pivotal ID # 179144780: Add json library (#427) Pivotal ID # 179834295: Excel dsl library Pivotal ID # 179356847: Use contanst as projects dependencies (#433) Pivotal ID # 179355330: Simplify submission files processing (#432) Pivotal ID # 179418565: Remove Option-Optional from backend (#438) Pivotal ID # 179728378: On behalf register user improvements (#446) Pivotal ID # 179436227: Improve Logs Messages (#447) Pivotal ID # 177839916: Add firemock container set-up (#443) Pivotal ID # 179784398: Improve all in one itest to include file list (#450) Pivotal ID # 179040289: BioStudies CLI Deployment (#452) Pivotal ID # 180124164: Re use ext mapping for file list (#464) Pivotal ID # 179939266: SpringBoot Admin App Registration (#461)

Feature Pivotal ID # 177042710: Support For Deleting Multiple Submissions (#394) Pivotal ID # 177839764: FIRE FileProcessingService (#401) Pivotal ID # 179055347: Update AE user activation email (#420) Pivotal ID # 178930818: Extended File List Endpoint (#418) Pivotal ID # 179363552: Add fire files interface source support (#436) Pivotal ID # 178461698: FIRE Database Persistence (#437) Pivotal ID # 179389139: FIRE Directories Persistence (#439) Pivotal ID # 179361117: Add support for groups administration (#434) Pivotal ID # 179609612: Submit Large Files To Fire (#444) Pivotal ID # 179418482: Persist PageTab FIRE (#440) Pivotal ID # 179513930: Add Warning For Empty File Lists (#448) Pivotal ID # 179040289: Async Migration (#451) Pivotal ID # 179922291: Inner File List Referenced Files (#457) Pivotal ID # 179810581: Files files storage (#456) Pivotal ID # 179922324: File Mode CLI Option (#465) Pivotal ID # 180152694: File List Validation (#466) Pivotal ID # 179921481: Fire ftp publishing (#467) Pivotal ID # 180253535: Submission storage mode (#468) Pivotal ID # 180282390: Reset password should activate user (#470) Pivotal Id # 180656091: Submission request should store all request information (#482) Pivotal ID # 180719965: Request should not save files list content (#484) Pivotal ID # 180850631: Stream serialization/deserilization support (#486) Pivotal ID # 180929480: Allow submission request to be manually re trigger (#493) Pivotal ID # 180481465: No attribute/empty attribute value handling (#490) Pivotal ID # 180962335: Only pending to process request need to be proccesed Pivotal ID # 180853446: Change RT ticket status upon submission (#496) Pivotal ID # 181061881: File list files should not be stored in submission document (#500) Pivotal ID # 181147744: Mongo migration with referenced submission in fileListFiles (#508) Pivotal ID # 181155613: Add push to partial updates exchange on study refresh (#506) Pivotal ID # 181179211 : Unactive draft when request is accepted (#507) Pivotal ID # 181154115: Extended submission, allow file list files loading option (#509) Pivotal ID # 180928073: Xml deserialization/serialization null support (#492) Pivotal ID # 180481465: Empty Attribute Value Handling (#513) Pivotal ID # 180640412: PMC Links Exporter (#483) Pivotal ID # 181330654: File List Empty Path Entries (#514) Pivotal ID # 181381956: Filter Active Requests For Existing Submissions (#516) Bugfix Pivotal ID # 178411427: Keyword search need to check root section title (#395) Pivotal ID # 178331250: Files Names Encoding (#431) Pivotal ID # 179435581: Deleting Submissions Isn't Working (#442) Pivotal ID # 179904973: Fix Notification URL (#455) Pivotal ID # 179867189: TSV Quotes Deserialization (#462) Pivotal ID # 180345449: handle empty title submission email notification (#472) Pivotal ID # 180446298: File list files are not being populated (#477) Pivotal ID # 180476322: File List URL Enconding (#478) Pivotal ID # 180957544: Re processing submissions is failing (#494) Pivotal ID # 181142221: Emails formating issues (#503) Chore Pivotal ID # 177839881: FIRE File Object Changes (#398) Pivotal ID # 178688699: Improve mongo index text (#403) Pivotal ID # 178213145: Activate ArrayExpress inactive users (#406) Pivotal ID # 178375625: Actuator Git Commit (#408) Pivotal ID # 179105055: Update Java Home (#421) Pivotal ID # 179144369: Add tsv library (#426) Pivotal ID # 178974842: Section Title Index (#416) Pivotal ID # 179144780: Add json library (#427) Pivotal ID # 179356847: Use contanst as projects dependencies (#433) Pivotal ID # 179355330: Simplify submission files processing (#432) Pivotal ID # 179418565: Remove Option-Optional from backend (#438) Pivotal ID # 179728378: On behalf register user improvements (#446) Pivotal ID # 179436227: Improve Logs Messages (#447) Pivotal ID # 177839916 Add firemock container set-up (#443) Pivotal ID # 179784398: Improve all in one itest to include file list (#450) Pivotal ID # 179040289: BioStudies CLI Deployment (#452) Pivotal ID # 179939266: SpringBoot Admin App Registration (#461) Pivotal ID # 179040289: Mongo Migration PROD (#479) Pivotal ID # 180762420: Dependencies upgrade (#485) Pivotal ID # 180891572: Immutable application properties (#487) Pivotal ID # 180924563: Remove sql submission support (#488) Pivotal ID # 180928080: Simplify refresh submission api (#491) Pivotal ID # 181032586: Remove submission-related changes from mysql set-up Pivotal ID # 181024221: Md5 should be calculated after accepting submission Pivotal ID # 181031416: Files size should be calculated after accepting submission Pivotal ID # 181127806: Enable rabbitmq_web_stomp in setup script Pivotal ID # 181196844: PMC task is failing to start (#510) Pivotal ID # 181076984: Submission Releaser Refactor (#512) Pivotal ID # 181206866: Remove application property enableMongo Pin RabbitMQ minor version number (#419)

BioStudies Backend Services v1.8 Feature Pivotal ID # 177042710: Support For Deleting Multiple Submissions (#394) Pivotal ID # 177839764: FIRE FileProcessingService (#401) Pivotal ID # 179055347: Update AE user activation email (#420) Pivotal ID # 178930818: Extended File List Endpoint (#418) Pivotal ID # 179363552: Add fire files interface source support (#436) Pivotal ID # 178461698: FIRE Database Persistence (#437) Pivotal ID # 179389139: FIRE Directories Persistence (#439) Pivotal ID # 179361117: Add support for groups administration (#434) Pivotal ID # 179609612: Submit Large Files To Fire (#444) Pivotal ID # 179418482: Persist PageTab FIRE (#440) Pivotal ID # 179513930: Add Warning For Empty File Lists (#448) Pivotal ID # 179040289: Async Migration (#451) Pivotal ID # 179922291: Inner File List Referenced Files (#457) Pivotal ID # 179810581: Files files storage (#456) Pivotal ID # 179922324: File Mode CLI Option (#465) Pivotal ID # 180152694: File List Validation (#466) Pivotal ID # 179921481: Fire ftp publishing (#467) Pivotal ID # 180253535: Submission storage mode (#468) Pivotal ID # 180282390: Reset password should activate user (#470) Pivotal Id # 180656091: Submission request should store all request information (#482) Pivotal ID # 180719965: Request should not save files list content (#484) Pivotal ID # 180850631: Stream serialization/deserilization support (#486) Pivotal ID # 180929480: Allow submission request to be manually re trigger (#493) Pivotal ID # 180481465: No attribute/empty attribute value handling (#490) Pivotal ID # 180962335: Only pending to process request need to be proccesed Pivotal ID # 180853446: Change RT ticket status upon submission (#496) Pivotal ID # 181061881: File list files should not be stored in submission document (#500) Pivotal ID # 181147744: Mongo migration with referenced submission in fileListFiles (#508) Pivotal ID # 181155613: Add push to partial updates exchange on study refresh (#506) Pivotal ID # 181179211 : Unactive draft when request is accepted (#507) Pivotal ID # 181154115: Extended submission, allow file list files loading option (#509) Pivotal ID # 180928073: Xml deserialization/serialization null support (#492) Pivotal ID # 180481465: Empty Attribute Value Handling (#513) Pivotal ID # 180640412: PMC Links Exporter (#483) Pivotal ID # 181330654: File List Empty Path Entries (#514) Pivotal ID # 181381956: Filter Active Requests For Existing Submissions (#516) Bugfix Pivotal ID # 178411427: Keyword searchs need to check root section title (#395) Pivotal ID # 178331250: Files Names Encoding (#431) Pivotal ID # 179435581: Deleting Submissions Isn't Working (#442) Pivotal ID # 179904973: Fix Notification URL (#455) Pivotal ID # 179867189: TSV Quotes Deserialization (#462) Pivotal ID # 180345449: handle empty title submission email notification (#472) Pivotal ID # 180446298: File list files are not being populated (#477) Pivotal ID # 180476322: File List URL Enconding (#478) Pivotal ID # 180957544: Re processing submissions is failing (#494) Pivotal ID # 181142221: Emails formating issues (#503) Chore Pivotal ID # 177839881: FIRE File Object Changes (#398) Pivotal ID # 178688699: Improve mongo index text (#403) Pivotal ID # 178213145: Activate ArrayExpress inactive users (#406) Pivotal ID # 178375625: Actuator Git Commit (#408) Pivotal ID # 179105055: Update Java Home (#421) Pivotal ID # 179144369: Add tsv library (#426) Pivotal ID # 178974842: Section Title Index (#416) Pivotal ID # 179144780: Add json library (#427) Pivotal ID # 179356847: Use contanst as projects dependencies (#433) Pivotal ID # 179355330: Simplify submission files processing (#432) Pivotal ID # 179418565: Remove Option-Optional from backend (#438) Pivotal ID # 179728378: On behalf register user improvements (#446) Pivotal ID # 179436227: Improve Logs Messages (#447) Pivotal ID # 177839916 Add firemock container set-up (#443) Pivotal ID # 179784398: Improve all in one itest to include file list (#450) Pivotal ID # 179040289: BioStudies CLI Deployment (#452) Pivotal ID # 179939266: SpringBoot Admin App Registration (#461) Pivotal ID # 179040289: Mongo Migration PROD (#479) Pivotal ID # 180762420: Dependencies upgrade (#485) Pivotal ID # 180891572: Immutable application properties (#487) Pivotal ID # 180924563: Remove sql submission support (#488) Pivotal ID # 180928080: Simplify refresh submission api (#491) Pivotal ID # 181032586: Remove submission-related changes from mysql set-up Pivotal ID # 181024221: Md5 should be calculated after accepting submission Pivotal ID # 181031416: Files size should be calculated after accepting submission Pivotal ID # 181127806: Enable rabbitmq_web_stomp in setup script Pivotal ID # 181196844: PMC task is failing to start (#510) Pivotal ID # 181076984: Submission Releaser Refactor (#512) Pivotal ID # 181206866: Remove application property enableMongo Pin RabbitMQ minor version number (#419)

Juan-EBI added 13 commits October 2, 2021 23:45

xml tests are missing

c9072bb

all in one tests done

7dd0530

code review changes

6ada633

code review changes

4b63c28

code review changes

e9dac16

code review changes

f480298

Merge branch 'master' of github.com-jcamilorada-EBI:EBIBioStudies/bio…

9e5fb0a

…studies-backend-services into feature/pivotal-179784398-Improve-all-in-one-itest-to-include-file-list

code review changes

50e7315

assertion pageTabs

96611a4

code review changes

ef2e7bd

code review changes

fd1ae59

updated config

d572a41

code review changes

b80211d

jhoanmanuelms approved these changes Oct 7, 2021

View reviewed changes

code review changes

e63a7d5

Juan-EBI merged commit 10c7582 into master Oct 7, 2021

Juan-EBI deleted the feature/pivotal-179784398-Improve-all-in-one-itest-to-include-file-list branch October 7, 2021 20:48

jhoanmanuelms mentioned this pull request Nov 25, 2021

Promote To BETA #474

Merged

jhoanmanuelms mentioned this pull request Mar 7, 2022

Promote To PROD #521

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Pivotal ID # 179784398: Improve all in one itest to include file list #450

Pivotal ID # 179784398: Improve all in one itest to include file list #450

Juan-EBI commented Oct 6, 2021

jhoanmanuelms left a comment

jhoanmanuelms Oct 7, 2021

Juan-EBI Oct 7, 2021

jhoanmanuelms Oct 7, 2021

Juan-EBI Oct 7, 2021 •

edited

Loading

jhoanmanuelms Oct 7, 2021

Pivotal ID # 179784398: Improve all in one itest to include file list #450

Pivotal ID # 179784398: Improve all in one itest to include file list #450

Conversation

Juan-EBI commented Oct 6, 2021

jhoanmanuelms left a comment

Choose a reason for hiding this comment

jhoanmanuelms Oct 7, 2021

Choose a reason for hiding this comment

Juan-EBI Oct 7, 2021

Choose a reason for hiding this comment

jhoanmanuelms Oct 7, 2021

Choose a reason for hiding this comment

Juan-EBI Oct 7, 2021 • edited Loading

Choose a reason for hiding this comment

jhoanmanuelms Oct 7, 2021

Choose a reason for hiding this comment

Juan-EBI Oct 7, 2021 •

edited

Loading