Handle incorrect CubeRegion/KeyValue usage by IMF

[zymon]

Here's short example:

import sdmx

imf = sdmx.Client("IMF")
imf.dataflow("1PI")

Fails with error:

--- SS without DSD ---
{1: False}

--- <class 'sdmx.message.StructureMessage'> ---
{2: <sdmx.StructureMessage>
  <Header>
    id: 'IREF437615'
    prepared: '2022-10-07T11:56:09+00:00'
    receiver: <Agency not_supplied>
    sender: <Agency IMF>
    source:
    test: False}

--- <class 'sdmx.model.AgencyScheme'> ---
{'AGENCIES': <AgencyScheme SDMX:AGENCIES(1.0) (16 items): SDMX Agency Scheme>}

--- <class 'sdmx.model.DataStructureDefinition'> ---
{'ECOFIN_DSD': <DataStructureDefinition IMF:ECOFIN_DSD(1.0): ECOFIN Data Structure Definition>}

--- <class 'sdmx.model.Agency'> ---
{'IMF': <Agency IMF>, 'SDMX': <Agency SDMX>}

--- <class 'sdmx.model.DataflowDefinition'> ---
{'1PI': <DataflowDefinition IMF:1PI(1.0): Coordinated Portfolio Investment Survey (CPIS) for Collection>}

--- <class 'sdmx.model.CategoryScheme'> ---
{'CS_SDMX_CENTRAL': <CategoryScheme IMF:CS_SDMX_CENTRAL(1.0) (1 items)>}

--- <class 'sdmx.model.Categorisation'> ---
{'b5979306-0fc2-8166-4370-fbff3bdd8189': <Categorisation IMF:b5979306-0fc2-8166-4370-fbff3bdd8189(1.0): Category 'IMF:CS_SDMX_CENTRAL(1.0).4' linking to Dataflow 'IMF:1PI(1.0)'>}

--- <class 'sdmx.model.Codelist'> ---
{'CL_DATADOMAIN': <Codelist IMF:CL_DATADOMAIN(1.0) (64 items): Data Domain>, 'CL_INDICATOR': <Codelist IMF:CL_INDICATOR(1.0) (91818 items): Economic Indicator>, 'CL_REF_AREA': <Codelist IMF:CL_REF_AREA(1.0) (709 items): Reference Area>, 'CL_AGE': <Codelist SDMX:CL_AGE(1.0) (5 items): Age>, 'CL_CONF_STATUS': <Codelist SDMX:CL_CONF_STATUS(1.0) (5 items): Confidentiality Status>, 'CL_DECIMALS': <Codelist SDMX:CL_DECIMALS(1.0) (16 items): Decimals>, 329904: <Codelist SDMX:CL_FREQ(1.0) (8 items): Frequency>, 329905: <Codelist SDMX:CL_FREQ(2.0) (9 items): Frequency>, 329977: <Codelist IMF:CL_OBS_STATUS(1.0) (19 items): Observation Status>, 329978: <Codelist SDMX:CL_OBS_STATUS(1.1) (17 items): Observation Status>, 'CL_OCCUPATION': <Codelist SDMX:CL_OCCUPATION(1.0) (619 items): Occupation>, 'CL_SEX': <Codelist SDMX:CL_SEX(2.0) (2 items): Sex>, 'CL_TIME_FORMAT': <Codelist SDMX:CL_TIME_FORMAT(1.0) (21 items): Time Format>, 332574: <Codelist IMF:CL_UNIT_MULT(1.0) (11 items): Unit Multiplier>, 332575: <Codelist SDMX:CL_UNIT_MULT(1.0) (9 items): Unit Multiplier>, 333332: <Codelist IMF:CL_OBS_STATUS(1.0) (0 items)>, 333333: <Codelist SDMX:CL_OBS_STATUS(1.1) (0 items)>, 333675: <Codelist IMF:CL_UNIT_MULT(1.0) (0 items)>, 333676: <Codelist SDMX:CL_UNIT_MULT(1.0) (0 items)>, 333727: <Codelist SDMX:CL_FREQ(2.0) (0 items)>, 333728: <Codelist SDMX:CL_FREQ(1.0) (0 items)>, 'CL_UNIT_MULT': <Codelist IMF:CL_UNIT_MULT(1.0) (0 items)>, 'CL_OBS_STATUS': <Codelist IMF:CL_OBS_STATUS(1.0) (0 items)>}

--- <class 'sdmx.model.ConceptScheme'> ---
{'ECOFIN_CONCEPTS': <ConceptScheme IMF:ECOFIN_CONCEPTS(1.0) (8 items): Concepts for EcoFin DSD>, 'CROSS_DOMAIN_CONCEPTS': <ConceptScheme SDMX:CROSS_DOMAIN_CONCEPTS(1.0) (135 items): SDMX Cross Domain Concept Scheme>}

--- Name ---
{333776: ('en', '1PI_CONSTRAINT')}

--- <class 'sdmx.reader.xml.Reference'> ---
{'1PI': <sdmx.reader.xml.Reference object at 0x000002484130DD60>}

--- <class 'sdmx.model.MemberSelection'> ---
{333779: <MemberSelection DATA_DOMAIN in {'1PI'}>, 334065: <MemberSelection INDICATOR in {'IAP_EUR', 'IAPDOF_S_XDC', 'IAPEOFM_XDC', 'IAPDOFI_L_EUR', 'IAPONH_EUR', 'IAPDSF_XDC', 'IAPUS_EUR', 'IAPDJY_S_XDC', 'IAPDA_S_USD', 'IAPDUS_XDC', 'IAPEUS_XDC', 'IAPONH_USD', 'IAPDA_S_EUR', 'IAPB_EUR', 'IAPDOFO_S_EUR', 'IAPDG_S_USD', 'IAPEOFM_USD', 'IAPDSF_L_XDC', 'IAPDOC_L_USD', 'ILPD_S_EUR', 'IAPDBP_L_XDC', 'IAPDONH_S_EUR', 'IAPOF_XDC', 'IAPDBP_EUR', 'IAPD_L_XDC', 'IAPDOC_L_EUR', 'IAPEOC_USD', 'IAPDOFO_EUR', 'IAPDUS_S_EUR', 'IAPON_EUR', 'IAPBP_USD', 'IAPDEU_L_USD', 'IAPDEU_S_USD', 'IAPDONO_L_USD', 'IAPDOF_XDC', 'IAPDONH_S_USD', 'IAPEEU_EUR', 'IAPDOC_L_XDC', 'IAPDUS_S_USD', 'IAPD_EUR', 'IAPDOF_S_USD', 'IAPONO_EUR', 'IAPDONH_L_XDC', 'IAPDJY_S_EUR', 'IAPDUS_S_XDC', 'IAPJY_EUR', 'IAPDON_S_XDC', 'IAPDBK_S_XDC', 'IAPEJY_EUR', 'IAPDUS_L_EUR', 'IAPD_S_EUR', 'ILPE_EUR', 'IAPEONN_EUR', 'IAPE_XDC', 'IAPG_EUR', 'IAPDBK_USD', 'IAPDBK_EUR', 'IAPDSF_S_XDC', 'IAPEONO_XDC', 'IAPG_XDC', 'IAPDBK_S_USD', 'IAPDONO_XDC', 'IAPDON_USD', 'IAPDBK_L_EUR', 'IAPEB_EUR', 'IAPDG_L_XDC', 'IAPDBK_XDC', 'IAPEUS_USD', 'IAPDONH_USD', 'IAPONO_XDC', 'IAPDG_L_EUR', 'IAPDONN_S_EUR', 'IAPDON_EUR', 'IAPD_S_USD', 'IAPDA_L_USD', 'IAPEOFO_XDC', 'IAPDJY_L_USD', 'IAPDJY_L_XDC', 'IAPA_USD', 'IAPDONN_S_XDC', 'IAPEOFO_USD', 'IAPDOFI_L_XDC', 'IAPDOFM_L_EUR', 'IAPUS_XDC', 'ILPD_S_USD', 'IAPEBP_XDC', 'IAP_USD', 'IAPDEU_USD', 'IAPEONO_EUR', 'IAPDON_XDC', 'IAPDJY_USD', 'IAPDBK_L_XDC', 'IAPEOFI_USD', 'IAPESF_XDC', 'IAPD_S_XDC', 'IAPDONN_S_USD', 'IAPEOFI_XDC', 'IAPONO_USD', 'IAPEBP_USD', 'IAPEG_XDC', 'IAPJY_USD', 'IAPEU_USD', 'IAPDEU_L_XDC', 'IAPDOF_S_EUR', 'IAPEONH_XDC', 'IAPDOFM_XDC', 'IAPDOFI_S_USD', 'IAPDONN_L_USD', 'IAPDEU_L_EUR', 'ILPD_USD', 'ILPD_EUR', 'IAPDOFM_USD', 'IAPD_XDC', 'IAPEON_XDC', 'IAPDONO_S_XDC', 'IAPEA_XDC', 'IAPOC_USD', 'IAPE_EUR', 'IAPDUS_L_USD', 'IAPDBK_S_EUR', 'IAPDONN_USD', 'ILPD_L_EUR', 'IAPDONN_EUR', 'IAPDSF_S_EUR', 'IAPONN_XDC', 'IAPDON_L_XDC', 'IAPE_USD', 'IAPDONH_L_EUR', 'ILP_XDC', 'IAPSF_USD', 'IAPDG_XDC', 'IAPOF_USD', 'IAPEON_EUR', 'IAPDG_S_EUR', 'IAPESF_USD', 'IAPDA_USD', 'IAPEEU_XDC', 'IAPDBP_S_EUR', 'IAPDEU_XDC', 'ILPD_XDC', 'IAPA_EUR', 'IAPDBP_S_USD', 'ILPD_S_XDC', 'IAPD_L_USD', 'IAPSF_XDC', 'IAPDUS_USD', 'ILP_USD', 'IAPJY_XDC', 'IAPDJY_L_EUR', 'IAPDOC_S_USD', 'IAPBP_XDC', 'IAPDA_EUR', 'IAPDOFI_EUR', 'IAPDJY_EUR', 'IAPOFI_XDC', 'IAPESF_EUR', 'IAPEUS_EUR', 'IAPEON_USD', 'IAPOFM_XDC', 'IAPDJY_S_USD', 'IAPDOFM_S_EUR', 'IAPEOFM_EUR', 'IAPDOFI_S_XDC', 'IAPDG_S_XDC', 'IAPDOFO_S_USD', 'IAPDONH_XDC', 'IAPDONO_L_EUR', 'IAPD_L_EUR', 'IAPDONN_XDC', 'IAPDONO_L_XDC', 'IAPEU_EUR', 'IAPDBP_USD', 'ILPE_XDC', 'IAPDOFM_EUR', 'IAPONH_XDC', 'IAPDOFO_USD', 'IAPDONH_L_USD', 'IAPDOF_EUR', 'IAPDONH_S_XDC', 'IAPDEU_S_XDC', 'IAPSF_EUR', 'IAPONN_EUR', 'IAPG_USD', 'IAPEG_USD', 'IAPDBP_L_USD', 'IAPOFM_USD', 'IAPOFI_USD', 'IAPDG_USD', 'IAPDOFO_S_XDC', 'IAPEBP_EUR', 'IAPDSF_S_USD', 'ILPE_USD', 'IAPOFO_EUR', 'IAPDEU_EUR', 'IAPEOF_EUR', 'IAPEU_XDC', 'IAPDOFO_XDC', 'IAPDONO_USD', 'IAPDG_L_USD', 'IAPEONH_EUR', 'IAPDONO_S_EUR', 'IAPOC_EUR', 'IAPDOC_XDC', 'IAPDBK_L_USD', 'IAPDJY_XDC', 'IAPDONO_EUR', 'IAPDOC_S_EUR', 'IAPEB_USD', 'IAPDON_S_USD', 'IAPDA_S_XDC', 'IAPDOFM_L_XDC', 'IAPDOF_L_EUR', 'IAP_XDC', 'IAPDA_L_XDC', 'IAPDON_L_USD', 'IAPEOFI_EUR', 'IAPDOC_EUR', 'IAPEEU_USD', 'IAPDONN_L_XDC', 'IAPDOFI_USD', 'IAPEOFO_EUR', 'IAPUS_USD', 'IAPOFO_XDC', 'IAPEONH_USD', 'IAPDOFO_L_XDC', 'IAPEOC_XDC', 'IAPDOF_L_XDC', 'IAPB_USD', 'IAPDG_EUR', 'IAPBP_EUR', 'IAPDOFI_L_USD', 'IAPEOF_XDC', 'IAPDA_L_EUR', 'IAPDEU_S_EUR', 'IAPDA_XDC', 'IAPEONN_XDC', 'IAPDONH_EUR', 'IAPDOC_USD', 'IAPDBP_XDC', 'IAPB_XDC', 'IAPEG_EUR', 'IAPDONN_L_EUR', 'IAPDOF_USD', 'IAPEA_USD', 'IAPA_XDC', 'IAPEJY_USD', 'IAPEA_EUR', 'IAPDOFM_L_USD', 'IAPON_USD', 'IAPDOFO_L_EUR', 'IAPDUS_L_XDC', 'IAPEOC_EUR', 'IAPONN_USD', 'IAPDSF_EUR', 'IAPOC_XDC', 'ILPD_L_XDC', 'IAPOFO_USD', 'IAPDOFI_S_EUR', 'IAPDBP_S_XDC', 'IAPD_USD', 'IAPDSF_USD', 'IAPDOFI_XDC', 'IAPDOFO_L_USD', 'IAPOF_EUR', 'IAPOFM_EUR', 'IAPDONO_S_USD', 'IAPDON_L_EUR', 'IAPDOFM_S_XDC', 'IAPEONN_USD', 'IAPDOFM_S_USD', 'IAPDOC_S_XDC', 'IAPDSF_L_EUR', 'IAPON_XDC', 'IAPOFI_EUR', 'IAPDSF_L_USD', 'IAPEONO_USD', 'IAPDBP_L_EUR', 'ILP_EUR', 'IAPDOF_L_USD', 'IAPEJY_XDC', 'IAPDON_S_EUR', 'IAPDUS_EUR', 'IAPEOF_USD', 'ILPD_L_USD', 'IAPEB_XDC'}>}

--- Value ---
{334066: 'PT1M', 334067: 'P6M', 334068: 'P3M', 334069: 'P1M', 334070: 'P1Y', 334071: 'P7D', 334072: 'P1D'}

<com:KeyValue xmlns:com="http://www.sdmx.org/resources/sdmxml/schemas/v2_1/common" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:mes="http://www.sdmx.org/resources/sdmxml/schemas/v2_1/message" xmlns:str="http://www.sdmx.org/resources/sdmxml/schemas/v2_1/structure" id="TIME_FORMAT">
  <com:Value/>
  <com:Value/>
  <com:Value/>
  <com:Value/>
  <com:Value/>
  <com:Value/>
  <com:Value/>
</com:KeyValue>

Traceback (most recent call last):
  File "r:\SharedLocal\ECFIN-DEV\fastop\python\.venv\lib\site-packages\sdmx\reader\xml.py", line 242, in read_message
    result = func(self, element)
  File "r:\SharedLocal\ECFIN-DEV\fastop\python\.venv\lib\site-packages\sdmx\reader\xml.py", line 1109, in _ms
    arg["values_for"] = cl.get(elem.attrib["id"])
  File "r:\SharedLocal\ECFIN-DEV\fastop\python\.venv\lib\site-packages\sdmx\model.py", line 879, in get
    raise KeyError(id)
KeyError: 'TIME_FORMAT'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "r:\SharedLocal\ECFIN-DEV\fastop\python\.venv\lib\site-packages\sdmx\client.py", line 496, in get
    msg = reader.read_message(response_content, dsd=kwargs.get("dsd", None))
  File "r:\SharedLocal\ECFIN-DEV\fastop\python\.venv\lib\site-packages\sdmx\reader\xml.py", line 259, in read_message
    raise XMLParseError from exc
sdmx.exceptions.XMLParseError: KeyError: 'TIME_FORMAT'

[khaeru]

Confirmed! Thanks for the report.

[khaeru]

Okay, on further investigation, this appears to be a quirk/error in the IMF data source, i.e. the SDMX-ML they provide does not conform to the standards. Strictly this is not a bug in sdmx1, which correctly implements this part of the standards.

I'm recording the details below for reference.

Maybe other implementations are more permissive for this particular quirk. I think we can add a special-case override to sdmx1 to allow parsing this message, and will open a PR to do this.

The details:

• The problematic XML is here.
• Within a <str:ContentConstraint><str:CubeRegion>, there are three <str:KeyValue> with the IDs DATA_DOMAIN, INDICATOR, and TIME_FORMAT.
• This constraint is attached to Dataflow=IMF:1PI(1.0), which in turn is structured by DataStructure=IMF:ECOFIN_DSD(1.0).
• In that data structure definition (DSD), DATA_DOMAIN and INDICATOR are dimensions, but TIME_FORMAT is an attribute (here).
• Here is where the SDMX-ML 2.1 standard defines the content of <str:DataCube>. Its contents can be <com:KeyValue> (defined here) or <com:Attribute> (defined here).
• In particular, the annotations read (emphasis added):

  CubeRegionKeyType is a type for providing a set of values for a dimension for the purpose of defining a data cube region. A set of distinct value can be provided, or if this dimension is represented as time, and time range can be specified.
  AttributeValueSetType defines the structure for providing values for a data attribute. If no values are provided, the attribute is implied to include/excluded from the region in which it is defined, with no regard to the value of the data attribute. Note that for metadata attributes which occur within other metadata attributes, a nested identifier can be provided. For example, a value of CONTACT.ADDRESS.STREET refers to the metadata attribute with the identifier STREET which exists in the ADDRESS metadata attribute in the CONTACT metadata attribute, which is defined at the root of the report structure.

• (This is not a SDMX-ML 3.0 message, but just for reference see here. The XML tag and schema names are different, but the distinction is the same.)

@zymon, while I work on the above-mentioned changes to the code, I'd encourage you to contact the data provider and kindly let them know about the error in their SDMX-ML content, so they are aware and can consider making a fix for it.

[zymon]

Hi @khaeru - thank you for checking. I've sent info to IMF, will let you know the response.

[zymon]

Thanks for the workaround for dataflow.
I've discovered another problem with the datastructure, here's example (using workaround #102)

import sdmx

imf = sdmx.Client("IMF")
df=imf.dataflow("1PI").dataflow["1PI"]
ds=imf.datastructure(df.structure.id)

Fails with error:

Work around incorrect use of CubeRegion/KeyValue in IMF StructureMessage; see https://github.com/khaeru/sdmx/issues/102
Work around incorrect use of CubeRegion/KeyValue in IMF StructureMessage; see https://github.com/khaeru/sdmx/issues/102
Work around incorrect use of CubeRegion/KeyValue in IMF StructureMessage; see https://github.com/khaeru/sdmx/issues/102
Work around incorrect use of CubeRegion/KeyValue in IMF StructureMessage; see https://github.com/khaeru/sdmx/issues/102
Work around incorrect use of CubeRegion/KeyValue in IMF StructureMessage; see https://github.com/khaeru/sdmx/issues/102
Work around incorrect use of CubeRegion/KeyValue in IMF StructureMessage; see https://github.com/khaeru/sdmx/issues/102
Work around incorrect use of CubeRegion/KeyValue in IMF StructureMessage; see https://github.com/khaeru/sdmx/issues/102
Work around incorrect use of CubeRegion/KeyValue in IMF StructureMessage; see https://github.com/khaeru/sdmx/issues/102
Work around incorrect use of CubeRegion/KeyValue in IMF StructureMessage; see https://github.com/khaeru/sdmx/issues/102
Work around incorrect use of CubeRegion/KeyValue in IMF StructureMessage; see https://github.com/khaeru/sdmx/issues/102
Work around incorrect use of CubeRegion/KeyValue in IMF StructureMessage; see https://github.com/khaeru/sdmx/issues/102



--- SS without DSD ---
{1: False}

--- <class 'sdmx.message.StructureMessage'> ---
{2: <sdmx.StructureMessage>
  <Header>
    id: 'IREF062861'
    prepared: '2022-10-11T16:38:01+00:00'
    receiver: <Agency not_supplied>
    sender: <Agency IMF>
    source:
    test: False}

--- <class 'sdmx.model.AgencyScheme'> ---
{'AGENCIES': <AgencyScheme SDMX:AGENCIES(1.0) (16 items): SDMX Agency Scheme>}

--- <class 'sdmx.model.DataStructureDefinition'> ---
{'ECOFIN_DSD': <DataStructureDefinition IMF:ECOFIN_DSD(1.0): ECOFIN Data Structure Definition>}

--- <class 'sdmx.model.Agency'> ---
{'IMF': <Agency IMF>, 'SDMX': <Agency SDMX>}

--- <class 'sdmx.model.DataflowDefinition'> ---
{'01R': <DataflowDefinition IMF:01R(1.0): Exchange Rates and International Reserves (01R) for Collection>, '1DI': <DataflowDefinition IMF:1DI(1.0): Coordinated Direct Investment Survey (CDIS) for Collection>, '1PI': <DataflowDefinition IMF:1PI(1.0): Coordinated Portfolio Investment Survey (CPIS) for Collection>, '1SR': <DataflowDefinition IMF:1SR(1.0): M&B: Central Bank (1SR) for Collection>, '2SR': <DataflowDefinition IMF:2SR(1.0): M&B: Other Depository Corporations (2SR) for Collection>, '4SR': <DataflowDefinition IMF:4SR(1.0): M&B: Other Financial Corporations (4SR) for Collection>, '5SR': <DataflowDefinition IMF:5SR(1.0): M&B: Monetary Aggregates (5SR) for Collection>, '60R': <DataflowDefinition IMF:60R(1.0): RES: Prices, Production, Labor and Trade (60R) for Collection>, '6SR': <DataflowDefinition IMF:6SR(1.0): M&B: Interest Rates and Share Prices (6SR) for Collection>, '90R': <DataflowDefinition IMF:90R(1.0): RES: National Accounts (90R) for Collection>, 'BCG': <DataflowDefinition IMF:BCG(1.0): Budgetary Central Government Operations>, 'BOP5_COLLECTION': <DataflowDefinition IMF:BOP5_COLLECTION(1.0): Balance of Payments (BPM5) for Collection>, 190: <DataflowDefinition IMF:BOP6(1.0): Balance of Payments (BPM6) for Collection>, 191: <DataflowDefinition IMF:BOP6(1.1): Balance of Payments (BPM6) for Collection>, 'BOP_BPM5': <DataflowDefinition IMF:BOP_BPM5(1.0): Balance of Payments -- BPM5>, 'BOP_BPM6': <DataflowDefinition IMF:BOP_BPM6(1.0): Balance of Payments -- BPM6>, 'CBS': <DataflowDefinition IMF:CBS(1.0): Central Bank Survey>, 'CBS_NSTD': <DataflowDefinition IMF:CBS_NSTD(1.0): Central Bank Survey_Non-Standard>, 'CGD': <DataflowDefinition IMF:CGD(1.0): Central Government debt (CGD)>, 'CGD_NSTD': <DataflowDefinition IMF:CGD_NSTD(1.0): Central Government Debt_Non-Standard>, 'CGO': <DataflowDefinition IMF:CGO(1.0): Central Government Operations (CGO)>, 'CGO_NSTD': <DataflowDefinition IMF:CGO_NSTD(1.0): Central government operations_Non-Standard>, 'COF': <DataflowDefinition IMF:COF(1.0): COFER: Currency Composition of Foreign Exchange Reserves>, 'CPI': <DataflowDefinition IMF:CPI(1.0): Consumer Price Index>, 'CPIH': <DataflowDefinition IMF:CPIH(1.0): Consumer Price Index Harmonized (CPIH) for collection>, 'DCS': <DataflowDefinition IMF:DCS(1.0): Depository Corporations Survey>, 'DCS_NSTD': <DataflowDefinition IMF:DCS_NSTD(1.0): Depository Corporations Survey_Non-Standard>, 'DOTS': <DataflowDefinition IMF:DOTS(1.0): Direction of Trade Statistics (DOTS) for Collection>, 'DSE': <DataflowDefinition IMF:DSE(1.0): Debt Securities>, 'EMP': <DataflowDefinition IMF:EMP(1.0): Employment>, 'EXD': <DataflowDefinition IMF:EXD(1.0): External debt>, 'EXD_BPM6': <DataflowDefinition IMF:EXD_BPM6(1.0): External Debt (EXD_BPM6)>, 'EXR': <DataflowDefinition IMF:EXR(1.0): Exchange rates>, 'FAS_COLLECTION': <DataflowDefinition IMF:FAS_COLLECTION(1.0): Financial Access Survey (FAS) for Collection>, 'FLI': <DataflowDefinition IMF:FLI(1.0): Forward looking Indicators>, 'FS1': <DataflowDefinition IMF:FS1(1.0): FSI: Institution of Deposit Takers and OFCs (FS1) for Collection>, 'FS2': <DataflowDefinition IMF:FS2(1.0): FSI: Sectoral Financial Statements (FS2) for Collection>, 'FSD': <DataflowDefinition IMF:FSD(1.0): FSI: Data Report Form (FSD) for Collection>, 'FSD_NSTD': <DataflowDefinition IMF:FSD_NSTD(1.0): FSI: Data Report Form_Non-Standard>, 'FSI': <DataflowDefinition IMF:FSI(1.0): Financial Soundness Indicators>, 'GFB': <DataflowDefinition IMF:GFB(1.0): GFS: Budgetary Central Govt: Operational Statement, Cash Statement, Balance Sheet (GFB) for Collection>, 'GFC': <DataflowDefinition IMF:GFC(1.0): GFS: Consolidated Central Govt: Operational Statement, Cash Statement, Balance Sheet (GFC) for Collection>, 'GFG': <DataflowDefinition IMF:GFG(1.0): GFS: Consolidated General Govt: Operational Statement, Cash Statement, Balance Sheet (GFG) for Collection>, 'GGD': <DataflowDefinition IMF:GGD(1.0): General Government gross debt (GGD)>, 'GGO': <DataflowDefinition IMF:GGO(1.0): General Government Operations (GGO)>, 'GYQ': <DataflowDefinition IMF:GYQ(1.0): GFS: Yearkbook Questionnaire (GYQ) for Collection>, 'IIP5_COLLECTION': <DataflowDefinition IMF:IIP5_COLLECTION(1.0): International Investment Position (BPM5) for Collection>, 'IIP6': <DataflowDefinition IMF:IIP6(1.0): International Investment Position (BPM6) for Collection>, 'IIP_BPM5': <DataflowDefinition IMF:IIP_BPM5(1.0): International Investment Position -- BPM5>, 'IIP_BPM6': <DataflowDefinition IMF:IIP_BPM6(1.0): International Investment Position -- BPM6>, 'ILV1': <DataflowDefinition IMF:ILV1(1.0): Official reserve assets>, 'ILV2': <DataflowDefinition IMF:ILV2(1.0): Template on International Reserves and Foreign Currency Liquidity>, 'IND': <DataflowDefinition IMF:IND(1.0): Index of Industrial Production>, 'IND_NSTD': <DataflowDefinition IMF:IND_NSTD(1.0): Index of Industrial Production_Non-Standard>, 'INF': <DataflowDefinition IMF:INF(1.0): Instrument Composition of Transactions in Foreign Exchange Reserves (INFER) for Collection>, 'INR': <DataflowDefinition IMF:INR(1.0): Interest rates>, 'INR_NSTD': <DataflowDefinition IMF:INR_NSTD(1.0): Interest rates_Non-Standard>, 'LMI': <DataflowDefinition IMF:LMI(1.0): Labor Market Indicators>, 'LMI_NSTD': <DataflowDefinition IMF:LMI_NSTD(1.0): Labor Market Indicators_Non-Standard>, 'MET': <DataflowDefinition IMF:MET(1.0): Merchandise trade>, 'MET_NSTD': <DataflowDefinition IMF:MET_NSTD(1.0): Merchandise Trade_Non-Standard>, 'MSG': <DataflowDefinition IMF:MSG(1.0): Monetary and Financial Statistics Aggregates>, 'NAG': <DataflowDefinition IMF:NAG(1.0): National Accounts>, 'NAG_08': <DataflowDefinition IMF:NAG_08(1.0): National Accounts -- 08'>, 'NAG_93': <DataflowDefinition IMF:NAG_93(1.0): National Accounts -- 93'>, 'NAG_NSTD': <DataflowDefinition IMF:NAG_NSTD(1.0): National Accounts - GDP_Non-Standard>, 'OFS': <DataflowDefinition IMF:OFS(1.0): Other Financial Corporations (OFC) survey>, 'POP': <DataflowDefinition IMF:POP(1.0): Population>, 'PPI': <DataflowDefinition IMF:PPI(1.0): Producer Price Index>, 'RTD': <DataflowDefinition IMF:RTD(1.0): Reserves Template for Collection>, 'SBS': <DataflowDefinition IMF:SBS(1.0): Accumulation Accounts (Sectoral Balance Sheets)>, 'SOC': <DataflowDefinition IMF:SOC(1.0): Socio-Demographic>, 'SPI': <DataflowDefinition IMF:SPI(1.0): Stock Market: Share Price Index>, 'UEM': <DataflowDefinition IMF:UEM(1.0): Unemployment>, 'WOE': <DataflowDefinition IMF:WOE(1.0): Wages and Earnings>, 'ZZZ_TEST_ECOFIN': <DataflowDefinition IMF:ZZZ_TEST_ECOFIN(1.0): Test Dataflow EcoFin DSD>, 'BOP6': <DataflowDefinition IMF:BOP6(1.0)>}

--- <class 'sdmx.model.Codelist'> ---
{'CL_DATADOMAIN': <Codelist IMF:CL_DATADOMAIN(1.0) (64 items): Data Domain>, 'CL_INDICATOR': <Codelist IMF:CL_INDICATOR(1.0) (91818 items): Economic Indicator>, 'CL_REF_AREA': <Codelist IMF:CL_REF_AREA(1.0) (709 items): Reference Area>, 'CL_AGE': <Codelist SDMX:CL_AGE(1.0) (5 items): Age>, 'CL_CONF_STATUS': <Codelist SDMX:CL_CONF_STATUS(1.0) (5 items): Confidentiality Status>, 'CL_DECIMALS': <Codelist SDMX:CL_DECIMALS(1.0) (16 items): Decimals>, 330181: <Codelist SDMX:CL_FREQ(1.0) (8 items): Frequency>, 330182: <Codelist SDMX:CL_FREQ(2.0) (9 items): Frequency>, 330254: <Codelist IMF:CL_OBS_STATUS(1.0) (19 items): Observation Status>, 330255: <Codelist SDMX:CL_OBS_STATUS(1.1) (17 items): Observation Status>, 'CL_OCCUPATION': <Codelist SDMX:CL_OCCUPATION(1.0) (619 items): Occupation>, 'CL_SEX': <Codelist SDMX:CL_SEX(2.0) (2 items): Sex>, 'CL_TIME_FORMAT': <Codelist SDMX:CL_TIME_FORMAT(1.0) (21 items): Time Format>, 332851: <Codelist IMF:CL_UNIT_MULT(1.0) (11 items): Unit Multiplier>, 332852: <Codelist SDMX:CL_UNIT_MULT(1.0) (9 items): Unit Multiplier>, 333609: <Codelist IMF:CL_OBS_STATUS(1.0) (0 items)>, 333610: <Codelist SDMX:CL_OBS_STATUS(1.1) (0 items)>, 333952: <Codelist IMF:CL_UNIT_MULT(1.0) (0 items)>, 333953: <Codelist SDMX:CL_UNIT_MULT(1.0) (0 items)>, 334004: <Codelist SDMX:CL_FREQ(2.0) (0 items)>, 334005: <Codelist SDMX:CL_FREQ(1.0) (0 items)>, 'CL_UNIT_MULT': <Codelist IMF:CL_UNIT_MULT(1.0) (0 items)>, 'CL_OBS_STATUS': <Codelist IMF:CL_OBS_STATUS(1.0) (0 items)>}

--- <class 'sdmx.model.ConceptScheme'> ---
{'ECOFIN_CONCEPTS': <ConceptScheme IMF:ECOFIN_CONCEPTS(1.0) (8 items): Concepts for EcoFin DSD>, 'CROSS_DOMAIN_CONCEPTS': <ConceptScheme SDMX:CROSS_DOMAIN_CONCEPTS(1.0) (135 items): SDMX Cross Domain Concept Scheme>}

--- <class 'sdmx.model.ContentConstraint'> ---
{'01R_CONSTRAINT': <ContentConstraint IMF:01R_CONSTRAINT(1.0): 01R_CONSTRAINT>, '1DI_CONSTRAINT': <ContentConstraint IMF:1DI_CONSTRAINT(1.0): 1DI_CONSTRAINT>, '1PI_CONSTRAINT': <ContentConstraint IMF:1PI_CONSTRAINT(1.0): 1PI_CONSTRAINT>, '1SR_CONSTRAINT': <ContentConstraint IMF:1SR_CONSTRAINT(1.0): 1SR_CONSTRAINT>, '2SR_CONSTRAINT': <ContentConstraint IMF:2SR_CONSTRAINT(1.0): 2SR_CONSTRAINT>, '4SR_CONSTRAINT': <ContentConstraint IMF:4SR_CONSTRAINT(1.0): 4SR_CONSTRAINT>, '5SR_CONSTRAINT': <ContentConstraint IMF:5SR_CONSTRAINT(1.0): 5SR_CONSTRAINT>, '60R_CONSTRAINT': <ContentConstraint IMF:60R_CONSTRAINT(1.0): 60R_CONSTRAINT>, '6SR_CONSTRAINT': <ContentConstraint IMF:6SR_CONSTRAINT(1.0): 6SR_CONSTRAINT>, '90R_CONSTRAINT': <ContentConstraint IMF:90R_CONSTRAINT(1.0): 90R_CONSTRAINT>, 'BCG_CONSTRAINT': <ContentConstraint IMF:BCG_CONSTRAINT(1.0): BCG_CONSTRAINT>, 'BOP5_CONSTRAINT': <ContentConstraint IMF:BOP5_CONSTRAINT(1.0): BOP5_CONSTRAINT>}

--- Name ---
{342930: ('en', 'BOP6_CONSTRAINT')}

--- <class 'sdmx.reader.xml.Reference'> ---
{'BOP6': <sdmx.reader.xml.Reference object at 0x000001E82526B790>}

--- Value ---
{342932: 'W1'}

<com:KeyValue xmlns:com="http://www.sdmx.org/resources/sdmxml/schemas/v2_1/common" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:mes="http://www.sdmx.org/resources/sdmxml/schemas/v2_1/message" xmlns:str="http://www.sdmx.org/resources/sdmxml/schemas/v2_1/structure" id="COUNTERPART_AREA">
  <com:Value/>
</com:KeyValue>

Traceback (most recent call last):
  File "r:\SharedLocal\ECFIN-DEV\fastop\python\.venv\lib\site-packages\sdmx\reader\xml.py", line 1143, in _ms
    cl, values_for = _ms_component(reader, elem, kind)
  File "r:\SharedLocal\ECFIN-DEV\fastop\python\.venv\lib\site-packages\sdmx\reader\xml.py", line 1113, in _ms_component
    return cl, cl.get(elem.attrib["id"])
  File "r:\SharedLocal\ECFIN-DEV\fastop\python\.venv\lib\site-packages\sdmx\model.py", line 879, in get
    raise KeyError(id)
KeyError: 'COUNTERPART_AREA'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "r:\SharedLocal\ECFIN-DEV\fastop\python\.venv\lib\site-packages\sdmx\reader\xml.py", line 242, in read_message
    result = func(self, element)
  File "r:\SharedLocal\ECFIN-DEV\fastop\python\.venv\lib\site-packages\sdmx\reader\xml.py", line 1152, in _ms
    cl, values_for = _ms_component(reader, elem, kinds["Attribute"])
  File "r:\SharedLocal\ECFIN-DEV\fastop\python\.venv\lib\site-packages\sdmx\reader\xml.py", line 1113, in _ms_component
    return cl, cl.get(elem.attrib["id"])
  File "r:\SharedLocal\ECFIN-DEV\fastop\python\.venv\lib\site-packages\sdmx\model.py", line 879, in get
    raise KeyError(id)
KeyError: 'COUNTERPART_AREA'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "r:\SharedLocal\ECFIN-DEV\fastop\python\.venv\lib\site-packages\sdmx\client.py", line 496, in get
    msg = reader.read_message(response_content, dsd=kwargs.get("dsd", None))
  File "r:\SharedLocal\ECFIN-DEV\fastop\python\.venv\lib\site-packages\sdmx\reader\xml.py", line 259, in read_message
    raise XMLParseError from exc
sdmx.exceptions.XMLParseError: KeyError: 'COUNTERPART_AREA'

[khaeru]

I've discovered another problem with the datastructure, here's example (using workaround #102)

Okay, thanks for the additional report. It's not clear at a glance whether this is the same or a distinct issue. I'll still publish the release and then investigate when I get a chance.