New sequence for : CFO-SWIM off nadir data

[antoinemerle]

Initial request

In the scope of the CFO Satellite : for the SWIM instruments :

we have been requested by METEO-FRANCE to generate BUFR data for the off-nadir part of the data.

This data needs to be disseminated through EUMETSAT to the member state.

At the moment the data is already implemented locally in EUM and with local code table.

Amendment details

context

Presentation attached :
SWIM data fot meteo france Off nadir data presentation to WMO.pdf

New Table B entries

first a more friendy way to visualise the sequence :

 - code: 001007
   description: satelliteIdentifier
 - code: 001031
  description: centre
 - code: 002019
  description: satelliteInstruments
 - code: 002020,
   description: satelliteClassification
- n_times : delayed replication
  value : 32
  table:
    - code: "005074"
      description: Index in range direction
    - code: "006029"
      description: Wave number
- n_times : fixed replication
  value : 24
  table:
    - code: "005075"
      description: Index in azimuthal direction
    - code: "025082"
      description: Azimuth angle
- code: "005037"
  description: Side track index or along lateral track row number
- code: "001124"
  description: Grid point identifier
- code: "005001"
  description: Latitude
- code: "006001"
  description: Longitude
- code: "004001"
  description: Year
- code: "004002"
  description: Month
- code: "004003"
  description: Day
- code: "004004"
  description: Hour
- code: "004005"
  description: Minute
- code: "004006"
  description: Second
- n_times : delayed Replication
  value : 32
    - code: "005074"
      description: Index in range direction
    - n_times : fixed replication
      value : 24
      table:
        - code: "005075"
          description: Index in azimuthal direction
        - code: "008085"
          description: Beam identifier
        - code: "022161"
          description: Wave spectra
        - code: "008085"
          description: Beam identifier
        - n_times : dealyed replication
          value : 32
          table:
            - code: "002134"
              description: Antenna beam azimuth
            - code: "022161"
              description: Wave spectra

Add the following entry to BUFR Table B:

Code	Element name	Units	Scale	Reference value	Data width
0 05 037	Side Track Index	Numeric	0	-1	11

New entries in existing code and flag tables

Code Table 0-08-085: Beam identifier

to view current table : https://vocabulary-manager.eumetsat.int/vocabularies/BUFR/WMO/41/TABLE_CODE_FLAG/008085

Action	Value	Meaning
Add	3	Combined average

New Table D entries

Add the following sequence 302065 "SWIM off-nadir data":

Table references	Element name	Note
301129	Observing satellite and instruments
1 04 000	Delayed replication of 4 descriptors	Repeated 32 times (currently), could be 40 for next IPF7
0 31 002	Extended Delayed descriptor replication factor	Variable: /nk
0 05 074	Index in range direction	Variable: /nk, Function: get_array_range_in_array(netCDF_keys)
2 02 133	Change scale from 1 to 6	Add 5 bits
006029	Wave number	Variable: /k_spectra, Function: to_subarrays(netCDF_keys) Wave number vector in m⁻¹ Unit: m⁻¹ Min: 0.0 Max: 1.0 Precision: 0.000001
2 02 000	Cancel change of scale
1 02 024	Replication of 2 descriptors	Repeated 24 times
005075	Index in azimuthal direction	Variable: /n_phi, Function: get_array_range_in_array(netCDF_keys) Number of n_phi vector
025082	Azimuth angle	Variable: /phi_vector, Function: to_subarrays(netCDF_keys) phi_vector(n_phi): Phi vector (center of azimuthal bin) Unit: degrees Min: 0.0 Max: 180.0 Precision: 0.1
2 01 120	Change width from 11 to 3
0 05 037	Side Track Index	Variable: /n_posneg, Function: apply_data_repetition([-1,1], False, netCDF_keys, None, False) Default value: python:[-1,1]*netCDF_keys 0 means -1 1 means +1
2 01 000	Cancel change of width
0 01 124	Grid point identifier	Variable: /n_box, Function: apply_data_repetition(netCDF_keys, True, None, 2, False) 1 to 528 for box number but can be higher or lower
0 05 001	Latitude	Variable: /lat_spec_l2, Function: get_reshaped_data(netCDF_keys, [a,b], [b*a]) Unit: degrees_north Min: -90.0 Max: 90.0 Precision: 0.001
0 06 001	Longitude (high accuracy)	Variable: /lon_spec_l2, Function: get_reshaped_data(netCDF_keys, [a,b], [b*a]) Unit: degrees_east Min: -180.0 Max: 180.0 Precision: 0.001
301130	High precision timestamp	Variable: /time_spec_l2 Functions: get_year_swim(netCDF_keys) get_month_swim(netCDF_keys) get_day_swim(netCDF_keys) get_hour_swim(netCDF_keys) get_minute_swim(netCDF_keys) get_second_swim(netCDF_keys) Ntim is the dimension of the time variable; it is equal to 2 because time is given as a pair: seconds and microseconds. Mean time of 2D spectrum area for each n_tim Unit: seconds since 2009-01-01 00:00:00 Min: 0 Max: 662,688,000 Precision: 1
1 10 000	Delayed replication of 10 descriptors	Repeated 32 times (currently), could be 40 for next IPF7
0 31 002	Extended Delayed Replication Factor
005074	Index in range direction	Variable: /nk
1 08 024	Replication of 8 descriptors	Repeated 24 times
005075	Index in azimuthal direction	Variable: n_phi, Function: get_array_range_in_array(netCDF_keys) Number of n_phi vector
008085	Beam identifier	Default value: 3 Note: Added entry for "Combined average" => 3
022161	Wave spectra	Variable: f_2D_combined, Function: get_reshaped_data(netCDF_keys, [a,b,c,d,e], [a*b, c*d])) For 2D mean height spectrum combining all spectral beams without ambiguity removal f_2D_combined(nk, n_phi, n_posneg, n_box) Unit: m⁴ Min: 0.0 Max: 1000.0 Precision: 0.0001
008085	Beam identifier	Default value: 7 Note: Set to missing (7)
1 02 000	Delayed replication of 2 descriptors
0 31 002	Extended delayed descriptor replication factor	Variable: /n_beam_l2, Default value: 3 Dimension of n_beam_l2: expected to be 1, but currently is 3
0 02 134	Antenna beam azimuth	Variable: /n_beam_l2, Function: get_antenna_beam_per_n_beam(netCDF_keys) N_beam_l2 defines the number of spectrum beams used to construct a wave spectrum. - Current configuration: beams at 6°, 8°, and 10°, so N_beam_l2 = 3. - Future configuration may use only 10°, so n_beam_l2 = 1. If n_beam is 1, value will be 10. If n_beam is 3, values will be 6, 8, and 10.
022161	Wave spectra	Variable: f2_D, Function: get_reshaped_data(netCDF_keys, [a,b,c,d,e], [a*b*e, c*d]) For 2D height spectrum for each spectral beam without ambiguity removal f2_D(nk, n_phi, n_posneg, n_box, n_beam_l2) Unit: m⁴ Min: 0.0 Max: 1000.0 Precision: 0.0001

where ``

Comments

No response

Requestor(s)

lotfi.aouf@meteo.fr
AOUF LOTFI (Scientifique)
Département Marine et Océanographie / R&D
Météo-France - DirOP/MAR/RD

Stakeholder(s)

EUMETSAT - MMDS
EUMETSAT - THOMAS Heinemann

Publication(s)

Impact on
table B : 1 new entry
Flag/code table : update existing entry
Table D : one new entry

Expected impact of change

None

Collaborators

Simon Elliott
Thomas Heinemann

References

No response

Validation

disclaimer : YOU NEED TO HAVE ECCODES 3.39 TO DECODE DATA (or a SW allowing nested replication)

REGRESS-SWIM-OGDR(2).zip

table local element from eccodes

005037|SideTrackIndexOrAlongLateralTrackRowNumber|Numeric|SideTrackIndexOrAlongLateralTrackRowNumber|Numeric|0|-1|11|Numeric|0|11

[amilan17]

https://github.com/wmo-im/et-data/wiki/2024.12.04.et.data notes:
Antoine introduced the proposal;

• still need to clarify whether satellite id should be included given that this is intended to be used as an additional sequence to an existing one as two different files:

[antoinemerle]

Dear @jbathegit , @SibylleK and @marijanacrepulja

Thanks again for your question and feedback.

I just spoke with Thomas Heinemann and the current implementation and I confirm this is well a different file that will be disseminated and populated.

I am going to add the sequence 301129 at the beginning of the new one I am requesting.

Data sample generation on going

Thanks again

[antoinemerle]

Dear team,

The data has been updated.
As suggested the code 301129 has been added and the data has been also updated accordingly ( here also for duplication).

• REGRESS-SWIM-OGDR(2).zip

Thanks a million for your support !

[amilan17]

https://github.com/wmo-im/et-data/wiki/2025.01.14.et.data notes:
Antoine is waiting for some feedback from MeteoFrance and prefers to have it introduced in THIS fast-track;
@antoinemerle will propose a sequence number; @antoinemerle will update the branch
@jbathegit can try to validate - hopefully the software can handle the two nested delayed replications; @sergioh-pessoal can help validate;

[antoinemerle]

Hi all ,

I updated the description of issue + uploaded the sequence in table D and here is the new data.

the data

And here is the data with the sequence D table :
SWIM-in-sequence-D.zip

sequence.def

"302065" = [ 301129, 104000, 031002, 005074, 202133, 006029, 202000, 102024, 005075, 025082, 201120, 005037, 201000, 001124, 005001, 006001, 301130, 110000, 031002, 005074, 108024, 005075, 008085, 022161, 008085, 102000, 031002, 002134, 022161 ]

element.table:

005037|SideTrackIndexOrAlongLateralTrackRowNumber|Numeric|SideTrackIndexOrAlongLateralTrackRowNumber|Numeric|0|-1|11|Numeric|0|11

[amilan17]

@antoinemerle - can you update the branch too?

[jbathegit]

I've been able to read the latest sample file provided by @antoinemerle, and I was very happy to see that our software can indeed handle triply-nested replication :-) However, and given that no reference key was included as to what the decoded output values should look like, I've provided my own gzip file for you on our public ftp server (go to https://ftp.emc.ncep.noaa.gov/mmb/jator2/for_AMerle/) which contains our decoded output, and which you can use to verify whether I obtained the expected values :-)

Other comments:

• Since this is intended to be part of the next FT2025-1, you'll need to use a master table version number of 44 when you exchange this data for real, since 44 will be the next release under FT2025-1.
• I see where you defined your new Table B element 0-05-037 to contain 11 bits, but then within your new Table D sequence 3-02-065 you immediately turn around and encapsulate the only occurrence of 0-05-037 with a Table C 2-01-120 operator to decrease its width by 8 bits. So why not just define the initial Table B element 0-05-037 to have 3 bits to begin with, instead of having to include an extra Table C operator within your Table D sequence?
• I only spot-checked a handful of the 1056 total data subsets in your sample file, but for all of the ones I did check I noticed that the 0-22-161 value that's sandwiched between the two occurrences of the 0-08-085 significance qualifier always seems to be "missing". Presumably the intent is to report some special value of 0-22-161 to which that significance qualifier applies, and which is separate from the replicated occurrences of 0-22-161 for the azimuths in the subsequent sequence. Again, maybe this was only true for the particular subsets that I checked, but if that particular 0-22-161 value is always going to be "missing", then why bother reporting it in the first place?

[amilan17]

as discussed @antoinemerle will get feedback next week and update the branch