unit test for the sentinel1-reader

[hfattahi]

This repository still lacks a unit test. I think writing the test is easy if we could have a Sentinel-1 SLC. One Sentinel-1 SLC is too big (4+ GB ) to store in the repo. We could download the data as part of the CI script but the problem is ASF and Scihub require credentials to download the data. What is the right approach to tackle this issue? Is there any sample Sentinel-1 SLC on ESA website that we could pull without credentials?

If there is no way around the data download issue, an alternative would be to upload only the xml files of one SLC to the repo, emulate the SAFE file data structure, zip it and use it for the test.

@yunjunz @LiangJYu @vbrancat @gshiroma @rtburns-jpl can you think of any other solution?

[yunjunz]

In circle CI, we could put the credentials as "Environment Variables", and pass it to the script, the value of the variables cannot be read or edited in the app once they are set.

[hfattahi]

In circle CI, we could put the credentials as "Environment Variables", and pass it to the script, the value of the variables cannot be read or edited in the app once they are set.

Awesome Yunjun. So that should solve the issue I guess. unless if we don't want to have the CI spend a lot of time downloading the entire SLC while we only need the metadata.

Perhaps we could have two tests, one that we would download the SLC and that could be a nightly/weekly test and a faster test which we could download only the xml files (e.g. using this) and as I suggested create the SAFE file data structure. This should be a much faster test to run.

[vbrancat]

Sorry for chiming in too late. Most of the time I download S1-A/B data using AWS buckets (e.g. http://sentinel1-slc-seasia-pds.s3-website-ap-southeast-1.amazonaws.com/datasets/slc/v1.1/). This should completely eliminate the need for username and password credentials but it requires coding a bit on s3 bucket interaction.

[hfattahi]

@vbrancat that's nice. I was not aware of this. So we can have multiple options for downloading data for a unit test!

[hfattahi]

Thanks @vbrancat I was not aware of this s3 bucket.

[vbrancat]

In circle CI, we could put the credentials as "Environment Variables", and pass it to the script, the value of the variables cannot be read or edited in the app once they are set.

Awesome Yunjun. So that should solve the issue I guess. unless if we don't want to have the CI spend a lot of time downloading the entire SLC while we only need the metadata.

Perhaps we could have two tests, one that we would download the SLC and that could be a nightly/weekly test and a faster test which we could download only the xml files (e.g. using this) and as I suggested create the SAFE file data structure. This should be a much faster test to run.

@hfattahi PR #18 should address the nightly test downloading S1-A/B data. For the second test, I am not quite sure what we want to do. To my understanding, we would like to download an annotation file, and maybe check that all the elements of the Sentinel1burst object are properly allocated?

[LiangJYu]

With #38 merged, can we close this PR?