You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Describe the bug
When running in csv-to-* (originally seen with csv-to-sqs), the records get added to the with the empty csv fields as "NaN". This is not proper json and causes json parse errors in stream-loader
To Reproduce
Steps to reproduce the behavior:
Use a docker-compose-demo demo yaml.
Edit the producer definition to use csv-to-sqs (rabbitMQ and Kafka should also see the problem)
Use the following input file for the producer: SENZING_INPUT_URL: /opt/senzing/g2/python/demo/truth/truthset-person-v1-set1-data.csv
Add the following to the producer service in the yaml file: volumes:${SENZING_G2_DIR:-/opt/senzing/g2}:/opt/senzing/g2
Describe the bug
When running in csv-to-* (originally seen with csv-to-sqs), the records get added to the with the empty csv fields as "NaN". This is not proper json and causes json parse errors in stream-loader
To Reproduce
Steps to reproduce the behavior:
docker-compose-demo
demo yaml.csv-to-sqs
(rabbitMQ and Kafka should also see the problem)SENZING_INPUT_URL: /opt/senzing/g2/python/demo/truth/truthset-person-v1-set1-data.csv
volumes:${SENZING_G2_DIR:-/opt/senzing/g2}:/opt/senzing/g2
stream-loader
. You should see errors like this:senzing-stream-loader | 2021-02-04 23:30:55,655 senzing-50010889E G2Engine.addRecord() G2ModuleGenericException: 30121E|JSON Parsing Failure [code=3,offset=114]; JSON: {"AMOUNT": 200, "CELL_PHONE_NUMBER": "982-122-2332", "DATA_SOURCE": "TEST", "DATE": "1/24/2018", "DATE_OF_BIRTH": NaN, "DRIVERS_LICENSE_NUMBER": NaN, "DRIVERS_LICENSE_STATE": NaN, "EMAIL_ADDRESS": "mickey@mmail.com", "ENTITY_TYPE": "GENERIC", "GENDER": NaN, "HOME_ADDR_CITY": "Andersonville", "HOME_ADDR_COUNTRY": NaN, "HOME_ADDR_LINE1": "P.O. Box 12987", "HOME_ADDR_POSTAL_CODE": "60611", "HOME_ADDR_STATE": "IL", "HOME_PHONE_NUMBER": NaN, "NATIONAL_ID_COUNTRY": NaN, "NATIONAL_ID_NUMBER": NaN, "PASSPORT_COUNTRY": NaN, "PASSPORT_NUMBER": NaN, "PRIMARY_NAME_FIRST": "Marie", "PRIMARY_NAME_LAST": "Sanchez", "PRIMARY_NAME_MIDDLE": NaN, "RECORD_ID": 1097, "SSN_LAST4": NaN, "SSN_NUMBER": NaN, "STATUS": "Active"}
Expected behavior
Load without errors.
The text was updated successfully, but these errors were encountered: