KeyError: 'STARTING_RANGE' due to expired numpy.float/int deprecation

[marcontk]

Description of the problem

I performed this process in isce2 and then used smallbaselineapp.py on MacOS for one month period, with zero errors. I repeated the same experiment on Linux, the difference from the previous experiment is that I used a longer time window (several months) and got the error below. What can it be?

Full script that generated the error

 smallbaselineApp.py LebuConsSenDT156.txt 

Full error message

(mintpy) smallbaselineApp.py LebuConsSenDT156.txt 

___________________________________________________________

  /##      /## /##             /##     /#######           
 | ###    /###|__/            | ##    | ##__  ##          
 | ####  /#### /## /#######  /######  | ##  \ ## /##   /##
 | ## ##/## ##| ##| ##__  ##|_  ##_/  | #######/| ##  | ##
 | ##  ###| ##| ##| ##  \ ##  | ##    | ##____/ | ##  | ##
 | ##\  # | ##| ##| ##  | ##  | ## /##| ##      | ##  | ##
 | ## \/  | ##| ##| ##  | ##  |  ####/| ##      |  #######
 |__/     |__/|__/|__/  |__/   \___/  |__/       \____  ##
                                                 /##  | ##
                                                |  ######/
   Miami InSAR Time-series software in Python    \______/ 
          MintPy 1.4.1, 2022-08-15
___________________________________________________________

--RUN-at-2022-12-09 13:17:16.835351--
Current directory: /home/cyclo/Insar/Processing/descending/LEBU_CONS/ISCE2/mintpy_lebu
Run routine processing with smallbaselineApp.py on steps: ['load_data', 'modify_network', 'reference_point', 'quick_overview', 'correct_unwrap_error', 'invert_network', 'correct_LOD', 'correct_SET', 'correct_troposphere', 'deramp', 'correct_topography', 'residual_RMS', 'reference_date', 'velocity', 'geocode', 'google_earth', 'hdfeos5']
Remaining steps: ['modify_network', 'reference_point', 'quick_overview', 'correct_unwrap_error', 'invert_network', 'correct_LOD', 'correct_SET', 'correct_troposphere', 'deramp', 'correct_topography', 'residual_RMS', 'reference_date', 'velocity', 'geocode', 'google_earth', 'hdfeos5']
--------------------------------------------------
Project name: LebuConsSenDT156
Go to work directory: /home/cyclo/Insar/Processing/descending/LEBU_CONS/ISCE2/mintpy_lebu
read custom template file: /home/cyclo/Insar/Processing/descending/LEBU_CONS/ISCE2/mintpy_lebu/LebuConsSenDT156.txt
update default template based on input custom template
No new option value found, skip updating /home/cyclo/Insar/Processing/descending/LEBU_CONS/ISCE2/mintpy_lebu/smallbaselineApp.cfg
read default template file: /home/cyclo/Insar/Processing/descending/LEBU_CONS/ISCE2/mintpy_lebu/smallbaselineApp.cfg


******************** step - load_data ********************

load_data.py --template /home/cyclo/Insar/Processing/descending/LEBU_CONS/ISCE2/mintpy_lebu/smallbaselineApp.cfg /home/cyclo/Insar/Processing/descending/LEBU_CONS/ISCE2/mintpy_lebu/LebuConsSenDT156.txt --project LebuConsSenDT156
processor : isce
SAR platform/sensor : Sen
--------------------------------------------------
prepare metadata files for isce products
prep_isce.py -m /home/cyclo/Insar/Processing/descending/LEBU_CONS/ISCE2/reference/IW2.xml -g /home/cyclo/Insar/Processing/descending/LEBU_CONS/ISCE2/merged/geom_reference -b /home/cyclo/Insar/Processing/descending/LEBU_CONS/ISCE2/baselines -f "/home/cyclo/Insar/Processing/descending/LEBU_CONS/ISCE2/merged/interferograms/*/filt_*.unw" --geom-files hgt.rdr lat.rdr lon.rdr los.rdr los.rdr
extract metadata from ISCE/topsStack xml file: /home/cyclo/Insar/Processing/descending/LEBU_CONS/ISCE2/reference/IW2.xml
/home/cyclo/miniconda3/envs/mintpy/lib/python3.10/site-packages/mintpy/load_data.py:785: UserWarning: prep_isce.py failed. Assuming its result exists and continue...
  warnings.warn('prep_isce.py failed. Assuming its result exists and continue...')
--------------------------------------------------
updateMode : True
compression: None
multilook x/ystep: 1/1
multilook method : nearest
--------------------------------------------------
searching geometry files info
input data files:
height          : /home/cyclo/Insar/Processing/descending/LEBU_CONS/ISCE2/merged/geom_reference/hgt.rdr
latitude        : /home/cyclo/Insar/Processing/descending/LEBU_CONS/ISCE2/merged/geom_reference/lat.rdr
longitude       : /home/cyclo/Insar/Processing/descending/LEBU_CONS/ISCE2/merged/geom_reference/lon.rdr
incidenceAngle  : /home/cyclo/Insar/Processing/descending/LEBU_CONS/ISCE2/merged/geom_reference/los.rdr
azimuthAngle    : /home/cyclo/Insar/Processing/descending/LEBU_CONS/ISCE2/merged/geom_reference/los.rdr
/home/cyclo/Insar/Processing/descending/LEBU_CONS/ISCE2/mintpy_lebu/inputs/geometryRadar.h5 exists, but can not read, remove it.
--------------------------------------------------
create HDF5 file /home/cyclo/Insar/Processing/descending/LEBU_CONS/ISCE2/mintpy_lebu/inputs/geometryRadar.h5 with w mode
create dataset /height             of <class 'numpy.float32'>   in size of (4975, 2956) with compression = lzf
create dataset /latitude           of <class 'numpy.float32'>   in size of (4975, 2956) with compression = lzf
create dataset /longitude          of <class 'numpy.float32'>   in size of (4975, 2956) with compression = lzf
create dataset /incidenceAngle     of <class 'numpy.float32'>   in size of (4975, 2956) with compression = lzf
create dataset /azimuthAngle       of <class 'numpy.float32'>   in size of (4975, 2956) with compression = lzf
prepare slantRangeDistance ...
Traceback (most recent call last):
  File "/home/cyclo/miniconda3/envs/mintpy/bin/smallbaselineApp.py", line 8, in <module>
    sys.exit(main())
  File "/home/cyclo/miniconda3/envs/mintpy/lib/python3.10/site-packages/mintpy/smallbaselineApp.py", line 1291, in main
    app.run(steps=inps.runSteps)
  File "/home/cyclo/miniconda3/envs/mintpy/lib/python3.10/site-packages/mintpy/smallbaselineApp.py", line 1047, in run
    self.run_load_data(sname)
  File "/home/cyclo/miniconda3/envs/mintpy/lib/python3.10/site-packages/mintpy/smallbaselineApp.py", line 356, in run_load_data
    mintpy.load_data.main(iargs)
  File "/home/cyclo/miniconda3/envs/mintpy/lib/python3.10/site-packages/mintpy/load_data.py", line 936, in main
    geom_radar_obj.write2hdf5(
  File "/home/cyclo/miniconda3/envs/mintpy/lib/python3.10/site-packages/mintpy/objects/stackDict.py", line 748, in write2hdf5
    data = self.get_slant_range_distance(box=box, xstep=xstep, ystep=ystep)
  File "/home/cyclo/miniconda3/envs/mintpy/lib/python3.10/site-packages/mintpy/objects/stackDict.py", line 504, in get_slant_range_distance
    data = ut.range_distance(self.extraMetadata,
  File "/home/cyclo/miniconda3/envs/mintpy/lib/python3.10/site-packages/mintpy/utils/utils0.py", line 53, in range_distance
    range_n, dR = float(atr['STARTING_RANGE']), float(atr['RANGE_PIXEL_SIZE'])
KeyError: 'STARTING_RANGE'

System information

• Operating system: Linux-OpenSuSE Leap 15.3 x86_64 GNU/Linux
• Python environment: conda
• MintPy version: 1.4.1, date 2022-08-15
• Your custom / default template file (if the bug is related to a specific dataset):
  LebuConsSenDT156.txt

[yunjunz]

I could think of 1) isce2 and mintpy are not installed in the same environment, or 2) the recently expired numpy.int/float deprecation #934. Updating your code to the latest development version is recommended. Could you re-try?

[marcontk]

Ok.

1. mintpy and isce2 I have them in different environments.

2. upgrading numpy to 1.24 had no solution because it had conflict with basemap

On the other hand a mintpy 1.5 upgrade with python3.10 and numpy1.24 gives the error:

AttributeError: module 'numpy' has no attribute 'float'. Did you mean: 'cfloat'?

Finally using Mintpy1.4.1 I downgraded the python version to 3.8.13 with numpy 1.23.3 and got nice results.

Thanks and Merry Christmas.

[yunjunz]

Thanks @marcontk. This confirms the error is due to the use of expired numpy.float/int deprecation since numpy 1.24, which has been fixed in #394, but not yet included in the released version. Installing mintpy in the development version and updating to the latest version should fix the issue.

We will cut a new release of mintpy soon after the holiday break, to fix this issue in the released version.

[gracebato]

It seems the problem is an incompatibility with python=3.10. I've tried re-installing using the development version as suggested but still get the KeyError: 'STARTING_RANGE' error. I then downgraded to python=3.8.0 with numpy=1.24, mintpy=1.5.0, and pyproj = 3.4.1 and was able to solve this issue.

[yunjunz]

The newly released version 1.5.1 is now available on the conda-forge channel. Cheers.