Possible error with reading ASF hyp3 azimuth angle data

[sar26]

Hi @yunjunz, @jhkennedy,

Raised from MintPy group conversation: https://groups.google.com/g/mintpy/c/ql2D1t8D-kw/m/yycRhtT3BQAJ

Using MintPy version 1.4.1, downloaded from conda.

Appears to be an issue with how the ASF hyp3 azimuth angle information is handled in MintPy.

I need to use asc_desc2horz_vert.py to resolve up-down, east-west motion for a large area, so I need to use the azimuth angle to get more accurate estimations. I followed the instructions in https://groups.google.com/g/mintpy/c/EcmuJU9jhX0/m/hmSoiv2XAQAJ and used the lv_phi and lv_theta files as the azAngleFile and incAngleFile in the configuration file.

If I calculate the motion without the geometry file, the results are:

asc_desc2horz_vert.py asc_velocity.h5 desc_velocity.h5

read  from file: asc_velocity.h5
calculate the constant LOS incidence / azimuth angles from metadata as:
LOS incidence angle: 37.6 deg
LOS azimuth   angle: 102.6 deg
read  from file: desc_velocity.h5
calculate the constant LOS incidence / azimuth angles from metadata as:
LOS incidence angle: 37.5 deg
LOS azimuth   angle: -102.6 deg

These values make sense and produce reasonable output results:

east-west data range: [-17.406893, 15.187981] mm/year
up-down data range: [-23.816603, 19.174559] mm/year

When running the same command but including the geometry files:

asc_desc2horz_vert.py asc_velocity.h5 desc_velocity.h5 -g asc_geometryGeo.h5 desc_geometryGeo.h5

The results are wrong:

east-west data range: [-739.0742, 695.91144] mm/year
up-down data range: [-232363660.0, 266020430.0] mm/year

Can this please be checked to see if there is an issue with dealing with the ASF hyp3 azimuth angle file in MintPy.

Thanks

Attached are plots of the azimuth files after the 'load_data' step.

[welcome]

👋 Thanks for opening your first issue here! Please filled out the template with as much details as possible. We appreciate that you took the time to contribute!
Make sure you read our contributing guidelines.

[yunjunz]

The azimuth angle in mintpy is following the isce2 convention (band2 of the los.rdr/geo file), which is defined as the azimuth angle of the LOS vector (from the ground to the SAR platform) measured from the north, with anti-clockwise direction as positive (note). Thus, a typical value for Sentinel-1 would be 102 deg for ascending, and -102 deg for descending orbits.

Since MintPy currently would read the lv_phi file as it is, as seen here. The plots above are very different from the +/- 102 deg value (as in the isce2 convention). @jhkennedy @forrestfwilliams could you check the definition of lv_phi in hyp3/gamma when you got a chance?

[jhkennedy]

@yunjunz we'll have a look

[cirrusasf]

In hyp3-gamma InSAR product, the LV_PHI is defined as the angle between the East direction and the projection of the look vector on the horizontal surface plane. Compare with the definition of azimuth angle in Mintpy, we can get the formula: LV_PHI_mintpy=3PI/2 +LV_PHI_hyp3

[yunjunz]

Here are two examples of azimuth angles I have, for your reference:

• Sentinel-1 ascending track 149
  

• Sentinel-1 descending track 156
  

Compare with the definition of azimuth angle in Mintpy, we can get the formula: LV_PHI_mintpy=3PI/2 +LV_PHI_hyp3

@cirrusasf This formula can not explain the value that @sar26 shown above.

[sar26]

@cirrusasf @yunjunz , I can send you the geometryGeo.h5 files if that helps? They are too large to attach here though.

[forrestfwilliams]

Hi @sar26, yes it would be helpful if we could look at the data you're using. Can you either post a link here or send it to ffwilliams2@alaska.edu?

[forrestfwilliams]

If you apply this conversion to your files, do you get reasonable angles?

import numpy as np

def convert_angle(hyp3_angle):
    mintpy_angle = np.rad2deg(hyp3_angle) - 90
    return mintpy_angle

[yunjunz]

Thank you @forrestfwilliams, your formula works! I will issue a PR for this fix.

[yunjunz]

Following the formula from @cirrusasf and @forrestfwilliams, I have updated the code in #865. But I don't have test data at hand, @sar26 could you try the PR and confirm if it works? Note that you need to remove your local *.rsc and geometryGeo.h5 files before re-running mintpy.

[cirrusasf]

@sar26 Can you tell us what pairs you are working on. The images (one ascending, one descending) you posted before look about right.

[SarahBrennand]

@cirrusasf I'm using T009A F1092 and T118D F679, the pairs are in the attached files.

@forrestfwilliams I'm in the process of copying the geometry data for you to look at. I'll send a google drive link to your email address.

@yunjunz I'm sorting out my data to test the changes you've made. I have to copy the input files from an archive as I delete them once I've loaded them to mintpy to save space.

T118D_F679_bw-15_ifg_pairs.txt
T009A_F1092_bw-15_all_ifg_pairs.txt

[SarahBrennand]

@yunjunz I'm having some problems testing the code, it could be to do with my setup, but not sure.

I've created a new conda environment and cloned your branch (to keep it separate from my running code). The HPC environment I use requires me to add the path to the mintpy directory into the scripts so it can find them when running PBS jobs. I used to use this setup with no problems, but recently changed to the conda version which removes this step and makes it easier to update versions.

So when running in the new conda environment, it just runs briefly and the ends with no screen outputs, errors or files. I tried copying the changed scripts to my existing conda environment, it starts reading the config file as expected but then produced an error:

    mintpy.geocode: auto --> no
    mintpy.save.kmz: auto --> no
Traceback (most recent call last):
  File "/scratch/project_mnt/S0008/anaconda3/pkgs/mintpy-1.4.1-pyhd8ed1ab_0/site-packages/mintpy/smallbaselineApp.py", line 1309, in <module>
    main(sys.argv[1:])
  File "/scratch/project_mnt/S0008/anaconda3/pkgs/mintpy-1.4.1-pyhd8ed1ab_0/site-packages/mintpy/smallbaselineApp.py", line 1290, in main
    app.open()
  File "/scratch/project_mnt/S0008/anaconda3/pkgs/mintpy-1.4.1-pyhd8ed1ab_0/site-packages/mintpy/smallbaselineApp.py", line 256, in open
    self._read_template()
  File "/scratch/project_mnt/S0008/anaconda3/pkgs/mintpy-1.4.1-pyhd8ed1ab_0/site-packages/mintpy/smallbaselineApp.py", line 310, in _read_template
    if tfile and  ut.run_or_skip(out_file=os.path.join(backup_dir, os.path.basename(tfile)),
TypeError: run_or_skip() got an unexpected keyword argument 'check_readable'

Perhaps I can copy a subset my input data files to Google Drive for you to test it on your setup?

[yunjunz]

Hi @SarahBrennand, I guess you installed both the released version of mintpy from conda-forge and setup the path to the developed version from my branch, this caused the path conflict. The solution is to install the developed version, following the installation on GitHub, which will install the mintpy dependencies (without mintpy itself) via conda, then install the mintpy itself (my branch in this case) using pip (pip install -e option is recommended). Note that you will need this installation approach after we tested and merged the PR also, since we won't make a new release right after it.

Perhaps I can copy a subset my input data files to Google Drive for you to test it on your setup?

Sure, that works too, I believe two pairs would be enough for testing. Thanks.

[SarahBrennand]

@yunjunz Thanks, I was missing the -e option in the pip install.

It runs successfully and the results are plotted below. The values are now closer in range as LOS azimuth calculated in my original post above (~ +/- 102 degrees).

T009A - data range: [-90.0, 105.2099] degree
azimuth angle (default plot)

T009A - data range: [103.805664, 105.2099] degree (masked -90 values)

T118D - data range: [-105.23892, -90.0] degree
azimuth angle (default plot)

T118D - data range: [-105.23892, -103.83663] degree (masked -90 values)

[yunjunz]

Thanks @SarahBrennand, these look all reasonable to me: the small variation from +/-102 deg is expected for different areas. It seems the no-data-value in lv_phi from hyp3 is 0, we should mask it out automatically for convenience. Could you also plot the incidenceAngle from lv_theta, so that I could check its no-data-value too?

[SarahBrennand]

@yunjunz, here are the incidence angle results:

T009A - data range: [30.422287, 90.0] degree
incidence angle (default plot)

T009A - data range: [30.422287, 45.446682] degree (masked 90 values)

T118D - data range: [30.397263, 90.0] degree
incidence angle (default plot)

I couldn't get the 90 degree values to mask properly for the descending data, so I haven't included a masked plot.

[yunjunz]

Thank you @SarahBrennand for the testing, this is helpful to validate the detail.

Close this issue as the PR (#865) is merged.