Script fails when a table column as upper case characters

[rap2hpoutre]

Script fails when a table column as upper case characters

Traceback (most recent call last):
  File "/usr/local/bin/pganonymize", line 8, in <module>
    sys.exit(main())
  File "/usr/local/lib/python3.9/site-packages/pganonymizer/__main__.py", line 10, in main
    main()
  File "/usr/local/lib/python3.9/site-packages/pganonymizer/cli.py", line 71, in main
    anonymize_tables(connection, schema.get('tables', []), verbose=args.verbose)
  File "/usr/local/lib/python3.9/site-packages/pganonymizer/utils.py", line 39, in anonymize_tables
    import_data(connection, column_dict, table_name, table_columns, primary_key, data)
  File "/usr/local/lib/python3.9/site-packages/pganonymizer/utils.py", line 140, in import_data
    copy_from(connection, data, 'source', table_columns)
  File "/usr/local/lib/python3.9/site-packages/pganonymizer/utils.py", line 119, in copy_from
    cursor.copy_from(new_data, table, sep=COPY_DB_DELIMITER, null='\\N', columns=columns)
psycopg2.errors.UndefinedColumn: column "createdat" of relation "source" does not exist

[hkage]

Hi, that means the column name in your database uses upper or mixed cased characters and you specified a lowercase column name in your YAML scheme? Or does this error occurs, even if you are using the same case in your scheme?

[bobslee]

Any info on this one? @rap2hpoutre

[rap2hpoutre]

@bobslee Oops, no sorry about that. I ended creating my own pganonymizer: https://github.com/rap2hpoutre/pg-anonymizer 😅

[hkage]

Sorry to hear that you can't use our project for your use case. I will try to improve it for those who want to use a Python based (and Python 2 supporting) tool.

We still don't use the anonymizer for our development databases therefore it takes some time and extra effort to improve this project. But I am more than happy if other contributors provide some fixes and enhancements.

[rap2hpoutre]

Thank you for your answer (and sorry for self-promotion, I can remove it!)

[hkage]

I made a fix that preserves the column and table names. That should fix issues with mixed or upper case names. As you described in your project, the anomyizer still cannot handle other PostgreSQL schemas than "public", but I will fix that in another commit.

You don't have to delete the link. But if have the time maybe you could try out the latest release of postgresql-anonymizer (I will upload it soon) and give me a short feedback if the error still occurs on your side.