SNOW-502598: Enable executing queries with asyncExec: true

[markandrus]

Description

My team is using the JavaScript Snowflake SDK. We need to be able to execute queries without waiting for the result. I see that the JavaScript Snowflake SDK has functionality for polling query results, a Result class, a RowStream class, etc.; however, it would be sufficient for us if we could just issue a request with asyncExec: true, get the Snowflake query ID back, and then monitor the query ourselves (for example, using RESULT_SCAN).

With this in mind, I've tried to create a small patch that does just that. This may partially address SNOW-502598 (see #486 and #208). I've also prepared a patch for @types/snowflake-sdk:

--- a/index.d.ts
+++ b/index.d.ts
@@ -571,7 +571,13 @@ export type Connection = NodeJS.EventEmitter & {
          * - {@link https://docs.snowflake.com/en/user-guide/nodejs-driver-use.html#fetching-data-types-as-strings Fetching Data Types As Strings}
          */
         fetchAsString?: Array<'String' | 'Boolean' | 'Number' | 'Date' | 'JSON' | 'Buffer'> | undefined;
-        complete?: (err: SnowflakeError | undefined, stmt: Statement, rows: any[] | undefined) => void;
+
+        /**
+         * Set to `true` to execute the query asynchronously, and the `complete` callback will receive an object with the query ID.
+         */
+        asyncExec?: boolean
+
+        complete?: (err: SnowflakeError | undefined, stmt: Statement, result: any[] | { queryId: string } | undefined) => void;
     }): Statement;
 
     /**

If you use Yarn v2 or later, you can try these changes out by

1. Updating your dependency to git@github.com:propeldata/snowflake-connector-nodejs.git#head=asyncExec
2. Using yarn patch to apply the @types/snowflake-sdk patch above.

Warning

This is definitely a hack, because the JavaScript Snowflake SDK's execute method is supposed to return a Statement object with APIs you can invoke, like getColumns and streamRows. These are probably not working with this patch; however, my team does not use these APIs anyway. We typically invoke the JavaScript Snowflake SDK as follows:

const { queryId } = new Promise((resolve, reject) => {
  client.execute({
    sqlText: 'CALL SYSTEM$WAIT(5, \'MINUTES\')',
    asyncExec: true,
    complete (err, _stmt, result) {
      if (err != null) {
        reject(err)
      } else {
        resolve(result)
      }
    }
  })
})

I would look to the JavaScript Snowflake SDK team for ideas on how to better integrate this functionality with the Statement object. Perhaps, when asyncExec: true is set, we should just synthesize a Result with a RowStream containing a singular row. That singular row could contain a singular column with the query ID. Maybe this would make sense?

Checklist

• Format code according to the existing code style (run npm run lint:check -- CHANGED_FILES and fix problems in changed code)
• Create tests which fail without the change (if possible)
• Make all tests (unit and integration) pass (npm run test:unit and npm run test:integration)
• Extend the README / documentation and ensure is properly displayed (if necessary)
• Provide JIRA issue id (if possible) or GitHub issue id in commit message

[sfc-gh-pmotacki]

@markandrus In version 1.9.1 of snowflake-sdk we added feature: "Added support for executing asynchronous queries".