Deploy ThoughtSpot meta data to Snowflake failed

[sae-bruce]

I ran the following command but returned an error.

C:\Users\m172485>cs_tools tools searchable deploy --connection-guid "1b8e9d0a-2701-4071-9bd5-0bc0dcc57b99" --database SAE200 --schema TS_METADATA --config sae_cloud --export C:\Users\m172485\Desktop
[09:09:25] INFO Attempting Basic authentication thoughtspot.py:213

        Status    Started At    Duration (s)   Task
       ────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
          ✅       01:09:26             8.75   Getting details for data source
          ❌       01:09:35             0.17   Customizing Searchable Worksheets to your environment
          🍿                                   Deploying the SpotApp to ThoughtSpot



      ┌──────────────────────────────────────────────────────── Traceback (most recent call last) ─────────────────────────────────────────────────────────┐
      │ C:\Users\m172485\AppData\Roaming\cs_tools\.cs_tools\lib\site-packages\cs_tools\cli\tools\searchable\app.py:93 in deploy                            │
      │                                                                                                                                                    │
      │    90 │   │   # Care for UPPERCASE or lowercase identity convention in dialects                                                                    │
      │    91 │   │   should_upper = "SNOWFLAKE" in dialect                                                                                                │
      │    92 │   │                                                                                                                                        │
      │ >  93 │   │   with tasks["customize_spotapp"]:                                                                                                     │
      │    94 │   │   │   here = pathlib.Path(__file__).parent                                                                                             │
      │    95 │   │   │   tmls = []                                                                                                                        │
      │    96                                                                                                                                              │
      │                                                                                                                                                    │
      │ C:\Users\m172485\AppData\Roaming\cs_tools\.cs_tools\lib\site-packages\cs_tools\cli\layout.py:115 in __exit__                                       │
      │                                                                                                                                                    │
      │   112 │   │   self.stop(error=bool(exc))                                                                                                           │
      │   113 │   │                                                                                                                                        │
      │   114 │   │   if exc is not None:                                                                                                                  │
      │ > 115 │   │   │   raise exc                                                                                                                        │
      │   116                                                                                                                                              │
      │   117                                                                                                                                              │
      │   118 class LiveTasks(Live):                                                                                                                       │
      │                                                                                                                                                    │
      │ C:\Users\m172485\AppData\Roaming\cs_tools\.cs_tools\lib\site-packages\cs_tools\cli\tools\searchable\app.py:99 in deploy                            │
      │                                                                                                                                                    │
      │    96 │   │   │                                                                                                                                    │
      │    97 │   │   │   for file in here.glob("**/*.tml"):                                                                                               │
      │    98 │   │   │   │   tml_cls = determine_tml_type(path=file)                                                                                      │
      │ >  99 │   │   │   │   tml = tml_cls.load(file)                                                                                                     │
      │   100 │   │   │   │                                                                                                                                │
      │   101 │   │   │   │   if isinstance(tml, Table):                                                                                                   │
      │   102 │   │   │   │   │   tml.table.db = database                                                                                                  │
      │                                                                                                                                                    │
      │ C:\Users\m172485\AppData\Roaming\cs_tools\.cs_tools\lib\site-packages\thoughtspot_tml\_tml.py:182 in load                                          │
      │                                                                                                                                                    │
      │   179 │   │   │   path = pathlib.Path(path)                                                                                                        │
      │   180 │   │                                                                                                                                        │
      │   181 │   │   try:                                                                                                                                 │
      │ > 182 │   │   │   instance = cls.loads(path.read_text())                                                                                           │
      │   183 │   │   except TMLDecodeError as e:                                                                                                          │
      │   184 │   │   │   e.path = path                                                                                                                    │
      │   185 │   │   │   raise e from None                                                                                                                │
      │                                                                                                                                                    │
      │ C:\Users\m172485\Anaconda3\envs\cs_tool\lib\pathlib.py:1135 in read_text                                                                           │
      │                                                                                                                                                    │
      │ C:\Users\m172485\Anaconda3\envs\cs_tool\lib\encodings\cp1252.py:23 in decode                                                                       │
      └────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┘
      UnicodeDecodeError: 'charmap' codec can't decode byte 0x81 in position 99956: character maps to <undefined>


                                         ┌───────────────────── 😲  This is an unhandled error!  😲 ─────────────────────┐
                                         │                                                                               │
                                         │ If you encounter this message more than once, please help by letting us know! │
                                         │                                                                               │
                                         │           GitHub: https://github.com/thoughtspot/cs_tools/issues/new/choose   │
                                         │                                                                               │
                                         └──────────── Run cs_tools logs report to send us your last error. ─────────────┘

C:\Users\m172485>

[boonhapus]

Hey @sae-bruce , I'm surprised you reached this one considering the TML files don't contain any special characters. Can you run the command again to its failure and then grab log files and share them with me?

cs_tools logs report --latest 5

https://thoughtspot.egnyte.com/ul/1bK0dz05L6

[sae-bruce]

cs tools get log error.txt
Hi @boonhapus , I ran the command "cs_tools logs report --latest 5" but got another error, please check the attached file.

Thanks and BR,
Bruce

[boonhapus]

Hey @sae-bruce , I'll have to fix this one but in the meantime you can grab the TML files here from Egnyte.

The Table TMLs need to be customized with the following details in order to import appropriately

• connection name
• database name
• schema name

.. all of the other files should reference the Tables and be OK to import. The TS_BI_SERVER.view.tml is not necessary if you are using a connected database like Snowflake, BigQuery, etc.

[sae-bruce]

Hi @boonhapus, I have downloaded the TML files and uploaded most of them to ThoughtSpot cloud.
But we don't have TS_GROUP_MEMBERSHIP and TS_ORG_MEMBERSHIP table, so we cannot upload TS_GROUP_MEMBERSHIP.table.tml, TS_ORG_MEMBERSHIP.table.tml and CS Tools ~ BI Server Advanced.worksheet.tml. TS_GROUP_MEMBERSHIP is used by CS Tools ~ BI Server Advanced worksheet.
Our purpose is to upload ThoughtSpot metadata to Snowflake, then import to Thoughtspot join them together and create worksheet and liveboard to check the ThoughtSpot adoption.
I already uploaded other tables to Snowflake except for TS_GROUP_MEMBERSHIP and TS_ORG_MEMBERSHIP, do you know how to upload these two tables to Snowflake?

Thanks and BR,
Bruce

[boonhapus]

@sae-bruce are you using searchable metadata with a https://thoughtspot.github.io/cs_tools/syncer/snowflake/? These tables should get automatically created in Snowflake in that case.

[sae-bruce]

Hi @boonhapus ,
I have figured it out, we don't have TS_GROUP_MEMBERSHIP and TS_ORG_MEMBERSHIP tables, but we have "TS_XREF_PRINCIPAL" & "TS_XREF_ORG", the schema of these tables are same, only names are different. So I have uploaded TS_GROUP_MEMBERSHIP.table.tml, TS_ORG_MEMBERSHIP.table.tml to ThoughtSpot successfully.

Thanks and BR,
Bruce

[boonhapus]

That's right, @sae-bruce . The database tables are named this way historically (because I'm bad at naming things 😄) and harder to rename in a backwards compatible manner. The metadata names are easy to rename as they simply point to external names.. so I took the opportunity to make them more user-friendly.

Glad we got you sorted out!