What does Data value corrupt: Not an array or missing the key "type"' mean?

[WolfgangFahl]

Just got the error message:

wikibaseintegrator.wbi_exceptions.ModificationFailed: 'Data value corrupt: Not an array or missing the key "type"'
'Data value corrupt: Not an array or missing the key "type"'

I have no clue what this is about. The code used to work with an older version of wikibaseintegrator and i am in the process of refactoring so i don't know whether my code or the changes in wikibaseintegrator caused this problem. I'd love to see a much more detailed error message with a hint on what to do in this case.

Here is the data sent:

{
  'aliases': {},
  'claims': {
    'P10692': [{
      'mainsnak': {
        'datatype': 'external-id',
        'datavalue': {},
        'property': 'P10692',
        'snaktype': 'value'
      },
      'rank': 'normal',
      'type': 'statement'
    }],
    'P1476': [{
      'mainsnak': {
        'datatype': 'monolingualtext',
        'datavalue': {
          'type': 'monolingualtext',
          'value': {
            'language': 'en',
            'text': '13th '
            'International '
            'Conference '
            'on '
            'Metadata '
            'and '
            'Semantics '
            'Research'
          }
        },
        'property': 'P1476',
        'snaktype': 'value'
      },
      'rank': 'normal',
      'type': 'statement'
    }],
    'P17': [{
      'mainsnak': {
        'datatype': 'wikibase-item',
        'datavalue': {
          'type': 'wikibase-entityid',
          'value': {
            'entity-type': 'item',
            'id': 'Q229',
            'numeric-id': 229
          }
        },
        'property': 'P17',
        'snaktype': 'value'
      },
      'rank': 'normal',
      'type': 'statement'
    }],
    'P179': [{
      'mainsnak': {
        'datatype': 'wikibase-item',
        'datavalue': {
          'type': 'wikibase-entityid',
          'value': {
            'entity-type': 'item',
            'id': 'Q105491179',
            'numeric-id': 105491179
          }
        },
        'property': 'P179',
        'snaktype': 'value'
      },
      'rank': 'normal',
      'type': 'statement'
    }],
    'P1813': [{
      'mainsnak': {
        'datatype': 'monolingualtext',
        'datavalue': {
          'type': 'monolingualtext',
          'value': {
            'language': 'en',
            'text': 'MTSR '
            '2019'
          }
        },
        'property': 'P1813',
        'snaktype': 'value'
      },
      'rank': 'normal',
      'type': 'statement'
    }],
    'P227': [{
      'mainsnak': {
        'datatype': 'external-id',
        'datavalue': {},
        'property': 'P227',
        'snaktype': 'value'
      },
      'rank': 'normal',
      'type': 'statement'
    }],
    'P276': [{
      'mainsnak': {
        'datatype': 'wikibase-item',
        'datavalue': {
          'type': 'wikibase-entityid',
          'value': {
            'entity-type': 'item',
            'id': 'Q185632',
            'numeric-id': 185632
          }
        },
        'property': 'P276',
        'snaktype': 'value'
      },
      'rank': 'normal',
      'type': 'statement'
    }],
    'P31': [{
      'mainsnak': {
        'datatype': 'wikibase-item',
        'datavalue': {
          'type': 'wikibase-entityid',
          'value': {
            'entity-type': 'item',
            'id': 'Q2020153',
            'numeric-id': 2020153
          }
        },
        'property': 'P31',
        'snaktype': 'value'
      },
      'rank': 'normal',
      'type': 'statement'
    }],
    'P5124': [{
      'mainsnak': {
        'datatype': 'external-id',
        'datavalue': {
          'type': 'string',
          'value': '88302'
        },
        'property': 'P5124',
        'snaktype': 'value'
      },
      'rank': 'normal',
      'type': 'statement'
    }],
    'P580': [{
      'mainsnak': {
        'datatype': 'time',
        'datavalue': {
          'type': 'time',
          'value': {
            'after': 0,
            'before': 0,
            'calendarmodel': 'http://www.wikidata.org/entity/Q1985727',
            'precision': 11,
            'time': '+2019-10-28T00:00:00Z',
            'timezone': 0
          }
        },
        'property': 'P580',
        'snaktype': 'value'
      },
      'rank': 'normal',
      'type': 'statement'
    }],
    'P582': [{
      'mainsnak': {
        'datatype': 'time',
        'datavalue': {
          'type': 'time',
          'value': {
            'after': 0,
            'before': 0,
            'calendarmodel': 'http://www.wikidata.org/entity/Q1985727',
            'precision': 11,
            'time': '+2019-10-31T00:00:00Z',
            'timezone': 0
          }
        },
        'property': 'P582',
        'snaktype': 'value'
      },
      'rank': 'normal',
      'type': 'statement'
    }],
    'P856': [{
      'mainsnak': {
        'datatype': 'url',
        'datavalue': {
          'type': 'string',
          'value': 'http://www.mtsr-conf.org/2019/home'
        },
        'property': 'P856',
        'snaktype': 'value'
      },
      'rank': 'normal',
      'type': 'statement'
    }],
    'P973': [{
      'mainsnak': {
        'datatype': 'url',
        'datavalue': {},
        'property': 'P973',
        'snaktype': 'value'
      },
      'rank': 'normal',
      'type': 'statement'
    }]
  },
  'descriptions': {
    'en': {
      'language': 'en',
      'value': 'MTSR 2019 Conference'
    }
  },
  'labels': {
    'en': {
      'language': 'en',
      'value': '13th International Conference on Metadata and '
      'Semantics Research'
    }
  },
  'type': 'item'
}

[LeMyst]

Hello @WolfgangFahl

Can you share how you generate this structure and how you send it to your Wikibase instance?

[WolfgangFahl]

We use https://wiki.bitplan.com/index.php/PyOnlineSpreadSheetEditing and https://wiki.bitplan.com/index.php/PyGenericSpreadSheet - with the corresponding github repositories.
An example input is https://docs.google.com/spreadsheets/d/194GV1-RK2VNe2hRI7fwdmeWCRPuqj5trkzgeVLMpMYc/edit#gid=1977978980

With the metadata mapping description in:

and the data in:

[dpriskorn]

Could you provide a code snippet that reproduce the issue?
BTW this error comes from Wikibase and there are multiple issues with the quality of those errors ie they don't tell you what you need to fix the problem.
See phabricator for details.

[WolfgangFahl]

https://github.com/WolfgangFahl/PyGenericSpreadSheet/blob/main/spreadsheet/wbquery.py and https://github.com/WolfgangFahl/PyGenericSpreadSheet/blob/main/spreadsheet/wikidata.py are core elements.

The refactoring is motivated by WolfgangFahl/pysotsog#46 where we intend to synchronize data from sementic mediawikis with wikidata using the same approach as in the spreadsheet case that we have been prototyping since 2022-05

[LeMyst]

@WolfgangFahl

I think your issue (partially?) come from the ExternalID without value :
'datavalue': {}

Or set the snaktype to NO_VALUE:
datatypes.ExternalID(prop_nr='P10692', snaktype=WikibaseSnakType.NO_VALUE)

I didn't test the others claims, but quickly looking at it, I think the P227 and P973 is in the same case.

[dpriskorn]

I created #496

[WolfgangFahl]

thx