Redshift Syncer throws an error when trying to INSERT INTO ts_metadata_object

[bmofkn]

We love the fact that the Redshit Syncer is now available! When trying it out earlier today, it was mostly successful. However, near the tail end of its INSERTS into Redshift, it failed.

We ran it via cs_tools tools searchable metadata --config prod_ts --syncer redshift://rssyncerDefinition.toml and after crunching through most of the data, it fails with the following message:

DataError: (psycopg2.errors.StringDataRightTruncation) value too long for type character varying(256)

FYI: We do not get this error when using the SQLite syncer

Perhaps this is helpful...some details from the logs report generated after the error:
[DEBUG - 2024-05-13T09:23:38-0600] [cs_tools.cli.commands.main - main.run 126] whoopsie, something went wrong! Traceback (most recent call last): File "/Users/redacted/Library/Application Support/cs_tools/.cs_tools/lib/python3.11/site-packages/sqlalchemy/engine/base.py", line 1967, in _exec_single_context self.dialect.do_execute( File "/Users/redacted/Library/Application Support/cs_tools/.cs_tools/lib/python3.11/site-packages/sqlalchemy/engine/default.py", line 924, in do_execute cursor.execute(statement, parameters) psycopg2.errors.StringDataRightTruncation: value too long for type character varying(256)

[boonhapus]

@bmofkn dang it! I thought I got all the fields that could cause this. Redshift TEXT columns have a hard limit to them. Would you mind shooting me an email so we can debug this one live? It'll be easier than sifting through logs exposed to the internet. 😁