-
Notifications
You must be signed in to change notification settings - Fork 127
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
GetArrowBatches panics when query returns 0 rows #1035
Comments
A simple fix might be func (scd *snowflakeChunkDownloader) getArrowBatches() []*ArrowBatch {
if scd.FirstBatch == nil || scd.FirstBatch.rec == nil {
return scd.ArrowBatches
}
return append([]*ArrowBatch{scd.FirstBatch}, scd.ArrowBatches...)
} |
hi and thank you for submitting this issue - there seems to be a discussion about this behaviour on #996 ; could you please take a look and see if it's relevant for you as well? |
@sfc-gh-dszmolka Actually yes, that issue seems relevant. My example errors out on a different line, but downgrading to the version mentioned in that issue does solve it... |
well the goal is not to downgrade the driver version ;) i thought the conversation on the referenced Issue could help tackling the issue (with the current version) by taking a different approach but let us take a look here if there's anything else for us to do here. |
we tested with 1.7.2 using took a look at the logs and
suggests you're using a query with bind variables, but without actually binding a value to could you please retry with 1.7.2 to see if it still fails when you give bind variable |
Sorry for the overly complex query, I just wanted a query that results in no rows. I updated the repo and tried replacing the query with INFO[0000]connection.go:388 gosnowflake.(*snowflakeConn).queryContextInternal Query: "SELECT 1 WHERE 1 = 0", []
DEBU[0000]log.go:148 gosnowflake.(*defaultLogger).Debugf empty qcc
INFO[0000]connection.go:118 gosnowflake.(*snowflakeConn).exec parameters: map[]
INFO[0000]connection.go:127 gosnowflake.(*snowflakeConn).exec bindings: map[]
INFO[0000]log.go:154 gosnowflake.(*defaultLogger).Infof params: &map[]
INFO[0000]retry.go:300 gosnowflake.(*retryHTTP).execute retryHTTP.totalTimeout: 0s
DEBU[0000]log.go:148 gosnowflake.(*defaultLogger).Debugf retry count: 0
INFO[0000]restful.go:252 gosnowflake.postRestfulQueryHelper postQuery: resp: &{200 OK 200 HTTP/1.1 1 1 map[Cache-Control:[no-cache, no-store] Connection:[keep-alive] Content-Type:[application/json] Date:[Tue, 23 Jan 2024 17:57:59 GMT] Expect-Ct:[enforce, max-age=3600] Strict-Transport-Security:[max-age=31536000] Vary:[Accept-Encoding, User-Agent] X-Content-Type-Options:[nosniff] X-Country:[Canada] X-Frame-Options:[deny] X-Xss-Protection:[1; mode=block]] 0xc00033db80 -1 [] false true map[] 0xc00075a200 0xc0000d8210}
INFO[0000]connection.go:157 gosnowflake.(*snowflakeConn).exec Success: true, Code: -1
DEBU[0000]log.go:148 gosnowflake.(*defaultLogger).Debugf adding query context: {0 45658026938507 0 }
INFO[0000]connection.go:180 gosnowflake.(*snowflakeConn).exec Exec/Query SUCCESS
INFO[0000]connection_util.go:141 gosnowflake.(*snowflakeConn).populateSessionParameters params: []gosnowflake.nameValueParameter{gosnowflake.nameValueParameter{Name:"TIMESTAMP_OUTPUT_FORMAT", Value:"YYYY-MM-DD HH24:MI:SS.FF3 TZHTZM"}, gosnowflake.nameValueParameter{Name:"CLIENT_PREFETCH_THREADS", Value:4}, gosnowflake.nameValueParameter{Name:"TIME_OUTPUT_FORMAT", Value:"HH24:MI:SS"}, gosnowflake.nameValueParameter{Name:"TIMESTAMP_TZ_OUTPUT_FORMAT", Value:""}, gosnowflake.nameValueParameter{Name:"CLIENT_RESULT_CHUNK_SIZE", Value:160}, gosnowflake.nameValueParameter{Name:"CLIENT_SESSION_KEEP_ALIVE", Value:false}, gosnowflake.nameValueParameter{Name:"CLIENT_METADATA_USE_SESSION_DATABASE", Value:false}, gosnowflake.nameValueParameter{Name:"CLIENT_OUT_OF_BAND_TELEMETRY_ENABLED", Value:false}, gosnowflake.nameValueParameter{Name:"QUERY_CONTEXT_CACHE_SIZE", Value:5}, gosnowflake.nameValueParameter{Name:"ENABLE_STAGE_S3_PRIVATELINK_FOR_US_EAST_1", Value:false}, gosnowflake.nameValueParameter{Name:"CLIENT_RESULT_PREFETCH_THREADS", Value:1}, gosnowflake.nameValueParameter{Name:"TIMESTAMP_NTZ_OUTPUT_FORMAT", Value:"YYYY-MM-DD HH24:MI:SS.FF3"}, gosnowflake.nameValueParameter{Name:"CLIENT_METADATA_REQUEST_USE_CONNECTION_CTX", Value:false}, gosnowflake.nameValueParameter{Name:"CLIENT_HONOR_CLIENT_TZ_FOR_TIMESTAMP_NTZ", Value:true}, gosnowflake.nameValueParameter{Name:"CLIENT_MEMORY_LIMIT", Value:1536}, gosnowflake.nameValueParameter{Name:"CLIENT_TIMESTAMP_TYPE_MAPPING", Value:"TIMESTAMP_LTZ"}, gosnowflake.nameValueParameter{Name:"TIMEZONE", Value:"America/Los_Angeles"}, gosnowflake.nameValueParameter{Name:"CLIENT_RESULT_PREFETCH_SLOTS", Value:2}, gosnowflake.nameValueParameter{Name:"CLIENT_TELEMETRY_ENABLED", Value:true}, gosnowflake.nameValueParameter{Name:"CLIENT_USE_V1_QUERY_API", Value:true}, gosnowflake.nameValueParameter{Name:"CLIENT_DISABLE_INCIDENTS", Value:true}, gosnowflake.nameValueParameter{Name:"CLIENT_RESULT_COLUMN_CASE_INSENSITIVE", Value:false}, gosnowflake.nameValueParameter{Name:"CSV_TIMESTAMP_FORMAT", Value:""}, gosnowflake.nameValueParameter{Name:"BINARY_OUTPUT_FORMAT", Value:"HEX"}, gosnowflake.nameValueParameter{Name:"CLIENT_ENABLE_LOG_INFO_STATEMENT_PARAMETERS", Value:false}, gosnowflake.nameValueParameter{Name:"CLIENT_TELEMETRY_SESSIONLESS_ENABLED", Value:true}, gosnowflake.nameValueParameter{Name:"CLIENT_FORCE_PROTECT_ID_TOKEN", Value:true}, gosnowflake.nameValueParameter{Name:"CLIENT_CONSENT_CACHE_ID_TOKEN", Value:false}, gosnowflake.nameValueParameter{Name:"DATE_OUTPUT_FORMAT", Value:"YYYY-MM-DD"}, gosnowflake.nameValueParameter{Name:"CLIENT_STAGE_ARRAY_BINDING_THRESHOLD", Value:65280}, gosnowflake.nameValueParameter{Name:"CLIENT_SESSION_KEEP_ALIVE_HEARTBEAT_FREQUENCY", Value:3600}, gosnowflake.nameValueParameter{Name:"AUTOCOMMIT", Value:true}, gosnowflake.nameValueParameter{Name:"CLIENT_SESSION_CLONE", Value:false}, gosnowflake.nameValueParameter{Name:"TIMESTAMP_LTZ_OUTPUT_FORMAT", Value:""}} LOG_SESSION_ID=14657069831386010
DEBU[0000]log.go:148 gosnowflake.(*defaultLogger).Debugf parameter. name: TIMESTAMP_OUTPUT_FORMAT, value: YYYY-MM-DD HH24:MI:SS.FF3 TZHTZM
DEBU[0000]log.go:148 gosnowflake.(*defaultLogger).Debugf parameter. name: CLIENT_PREFETCH_THREADS, value: 4
DEBU[0000]log.go:148 gosnowflake.(*defaultLogger).Debugf parameter. name: TIME_OUTPUT_FORMAT, value: HH24:MI:SS
DEBU[0000]log.go:148 gosnowflake.(*defaultLogger).Debugf parameter. name: TIMESTAMP_TZ_OUTPUT_FORMAT, value:
DEBU[0000]log.go:148 gosnowflake.(*defaultLogger).Debugf parameter. name: CLIENT_RESULT_CHUNK_SIZE, value: 160
DEBU[0000]log.go:148 gosnowflake.(*defaultLogger).Debugf parameter. name: CLIENT_SESSION_KEEP_ALIVE, value: false
DEBU[0000]log.go:148 gosnowflake.(*defaultLogger).Debugf parameter. name: CLIENT_METADATA_USE_SESSION_DATABASE, value: false
DEBU[0000]log.go:148 gosnowflake.(*defaultLogger).Debugf parameter. name: CLIENT_OUT_OF_BAND_TELEMETRY_ENABLED, value: false
DEBU[0000]log.go:148 gosnowflake.(*defaultLogger).Debugf parameter. name: QUERY_CONTEXT_CACHE_SIZE, value: 5
DEBU[0000]log.go:148 gosnowflake.(*defaultLogger).Debugf parameter. name: ENABLE_STAGE_S3_PRIVATELINK_FOR_US_EAST_1, value: false
DEBU[0000]log.go:148 gosnowflake.(*defaultLogger).Debugf parameter. name: CLIENT_RESULT_PREFETCH_THREADS, value: 1
DEBU[0000]log.go:148 gosnowflake.(*defaultLogger).Debugf parameter. name: TIMESTAMP_NTZ_OUTPUT_FORMAT, value: YYYY-MM-DD HH24:MI:SS.FF3
DEBU[0000]log.go:148 gosnowflake.(*defaultLogger).Debugf parameter. name: CLIENT_METADATA_REQUEST_USE_CONNECTION_CTX, value: false
DEBU[0000]log.go:148 gosnowflake.(*defaultLogger).Debugf parameter. name: CLIENT_HONOR_CLIENT_TZ_FOR_TIMESTAMP_NTZ, value: true
DEBU[0000]log.go:148 gosnowflake.(*defaultLogger).Debugf parameter. name: CLIENT_MEMORY_LIMIT, value: 1536
DEBU[0000]log.go:148 gosnowflake.(*defaultLogger).Debugf parameter. name: CLIENT_TIMESTAMP_TYPE_MAPPING, value: TIMESTAMP_LTZ
DEBU[0000]log.go:148 gosnowflake.(*defaultLogger).Debugf parameter. name: TIMEZONE, value: America/Los_Angeles
DEBU[0000]log.go:148 gosnowflake.(*defaultLogger).Debugf parameter. name: CLIENT_RESULT_PREFETCH_SLOTS, value: 2
DEBU[0000]log.go:148 gosnowflake.(*defaultLogger).Debugf parameter. name: CLIENT_TELEMETRY_ENABLED, value: true
DEBU[0000]log.go:148 gosnowflake.(*defaultLogger).Debugf parameter. name: CLIENT_USE_V1_QUERY_API, value: true
DEBU[0000]log.go:148 gosnowflake.(*defaultLogger).Debugf parameter. name: CLIENT_DISABLE_INCIDENTS, value: true
DEBU[0000]log.go:148 gosnowflake.(*defaultLogger).Debugf parameter. name: CLIENT_RESULT_COLUMN_CASE_INSENSITIVE, value: false
DEBU[0000]log.go:148 gosnowflake.(*defaultLogger).Debugf parameter. name: CSV_TIMESTAMP_FORMAT, value:
DEBU[0000]log.go:148 gosnowflake.(*defaultLogger).Debugf parameter. name: BINARY_OUTPUT_FORMAT, value: HEX
DEBU[0000]log.go:148 gosnowflake.(*defaultLogger).Debugf parameter. name: CLIENT_ENABLE_LOG_INFO_STATEMENT_PARAMETERS, value: false
DEBU[0000]log.go:148 gosnowflake.(*defaultLogger).Debugf parameter. name: CLIENT_TELEMETRY_SESSIONLESS_ENABLED, value: true
DEBU[0000]log.go:148 gosnowflake.(*defaultLogger).Debugf parameter. name: CLIENT_FORCE_PROTECT_ID_TOKEN, value: true
DEBU[0000]log.go:148 gosnowflake.(*defaultLogger).Debugf parameter. name: CLIENT_CONSENT_CACHE_ID_TOKEN, value: false
DEBU[0000]log.go:148 gosnowflake.(*defaultLogger).Debugf parameter. name: DATE_OUTPUT_FORMAT, value: YYYY-MM-DD
DEBU[0000]log.go:148 gosnowflake.(*defaultLogger).Debugf parameter. name: CLIENT_STAGE_ARRAY_BINDING_THRESHOLD, value: 65280
DEBU[0000]log.go:148 gosnowflake.(*defaultLogger).Debugf parameter. name: CLIENT_SESSION_KEEP_ALIVE_HEARTBEAT_FREQUENCY, value: 3600
DEBU[0000]log.go:148 gosnowflake.(*defaultLogger).Debugf parameter. name: AUTOCOMMIT, value: true
DEBU[0000]log.go:148 gosnowflake.(*defaultLogger).Debugf parameter. name: CLIENT_SESSION_CLONE, value: false
DEBU[0000]log.go:148 gosnowflake.(*defaultLogger).Debugf parameter. name: TIMESTAMP_LTZ_OUTPUT_FORMAT, value:
DEBU[0000]rows.go:78 gosnowflake.(*snowflakeRows).Close Rows.Close LOG_SESSION_ID=14657069831386010
INFO[0000]connection.go:268 gosnowflake.(*snowflakeConn).Close Close LOG_SESSION_ID=14657069831386010
DEBU[0000]log.go:210 gosnowflake.(*defaultLogger).Debug nothing to send to telemetry
INFO[0000]restful.go:326 gosnowflake.closeSession close session LOG_SESSION_ID=14657069831386010
INFO[0000]retry.go:300 gosnowflake.(*retryHTTP).execute retryHTTP.totalTimeout: 5s LOG_SESSION_ID=14657069831386010
DEBU[0000]log.go:148 gosnowflake.(*defaultLogger).Debugf retry count: 0
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x1026139]
goroutine 1 [running]:
github.com/snowflakedb/gosnowflake.(*snowflakeChunkDownloader).getArrowBatches(0xc000667e10?)
github.com/snowflakedb/gosnowflake@v1.7.2/chunk_downloader.go:252 +0x19
github.com/snowflakedb/gosnowflake.(*snowflakeRows).GetArrowBatches(0xc0000da0d0?)
github.com/snowflakedb/gosnowflake@v1.7.2/rows.go:171 +0x162
main.main()
github.com/vtan-fortinet/musical-octo-funicular/main.go:66 +0x306 query id |
thanks to your inputs, the issue has been found - and already fixed by the dev team: #1039 Since PR is merged, it can be already tested with installing the |
fix released with gosnowflake 1.8.0 |
Please answer these questions before submitting your issue.
In order to accurately debug the issue this information is required. Thanks!
What version of GO driver are you using?
github.com/snowflakedb/gosnowflake v1.7.2
What operating system and processor architecture are you using?
Linux hostname 5.15.0-91-generic #101~20.04.1-Ubuntu SMP Thu Nov 16 14:22:28 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux Distributor ID: Ubuntu Description: Ubuntu 20.04.6 LTS Release: 20.04 Codename: focal
run
go version
in your consolego version go1.21.6 linux/amd64
4.Server version: 8.2.0
You may get the server version by running a query:
Minimal reproducible example here: https://github.com/vtan-fortinet/musical-octo-funicular
I've bisected the issue to commit a4c3557
What did you expect to see?
GetArrowBatches() should have returned an empty array, and not panic.
Can you set logging to DEBUG and collect the logs?
ytb20147
The text was updated successfully, but these errors were encountered: