Nil pointer access due to empty chunk returned from gosnowflake.buildFirstArrowChunk

[till-kolditz]

panic: runtime error: invalid memory address or nil pointer dereference in gosnowflake due to the following sequence of events:
In gosnowflake.(*snowflakeChunkDownloader).start() the chunk is created using buildFirstArrowChunk and then decodeArrowChunk is called on it.
When an error happens during creation of the chunk, buildFirstArrowChunk silently swallows any error and returns a default-constructed chunk. By that, the chunk's reader pointer is nil and when calling Next() on it in arrow_chunk.go:26 we get a nil pointer dereference in github.com/apache/arrow/go/v12/arrow/ipc.(*Reader).Next(0x0?)

Stack trace (sensitive parts removed):

panic({0x1c676e0, 0x41805c0})
        GOROOT/src/runtime/panic.go:884 +0x212
github.com/apache/arrow/go/v12/arrow/ipc.(*Reader).Next(0x0?)
       vendor/github.com/apache/arrow/go/v12/arrow/ipc/reader.go:167 +0x18
gosnowflake.(*arrowResultChunk).decodeArrowChunk(0xc008195cb8, {0xc007d7a780, 0x1, 0x34743e0?}, 0xe8?)
        gosnowflake/arrow_chunk.go:26 +0xd7
gosnowflake.(*snowflakeChunkDownloader).start(0xc010270480)
        gosnowflake/chunk_downloader.go:114 +0x225
gosnowflake.(*snowflakeConn).queryContextInternal(0xc00fb50c60, {0x3478338, 0xc009c3a060}, {0xc00f220000, 0x151}, {0xc00f1ee500, 0x8, 0x8})
        gosnowflake/connection.go:420 +0x549
gosnowflake.(*snowflakeConn).QueryContext(0xc00dc895b0?, {0x3478338, 0xc009c3a060}, {0xc00f220000, 0x151}, {0xc00f1ee500, 0x8, 0x8})
        gosnowflake/connection.go:350 +0x13a

1. What version of GO driver are you using?

5c79db8
We required two fixes included since 1.7.0.

2. What operating system and processor architecture are you using?

Ubuntu 20.04.6
x86_64

3. What version of GO are you using?

go1.20.10 linux/amd64

4.Server version:* E.g. 1.90.1
You may get the server version by running a query:

7.4

5. What did you do?

6. What did you expect to see?

An error must be returned from chunk creation all the way up to the client code.

7. Can you set logging to DEBUG and collect the logs?

   no

8. What is your Snowflake account identifier, if any? (Optional)

[sfc-gh-dszmolka]

thank you for raising this issue, we'll take a look

[sfc-gh-dszmolka]

is there perhaps any chance you have a reproducible scenario ? i tried breaking the response / chunk to see if i can get to the panic, but obviously doing something wrong because i saw different results with the latest code in main.
attempts:

• delete response body:

ERRO[0055]log.go:178 gosnowflake.(*defaultLogger).Errorf failed to extract HTTP response body. 
URL: https://my-customer-stage.s3.eu-central-1.amazonaws.com/stageid/results/01b0cef8-0102-25e0-0002-4776000d7012_0/main/data_0_0_0?.. 
err: EOF 

• delete part of response body:

ERRO[0152]log.go:178 gosnowflake.(*defaultLogger).Errorf failed to extract HTTP response body. 
URL: https://my-customer-stage.s3.eu-central-1.amazonaws.com/stageid/results/01b0cef8-0102-25e0-0002-4776000d7012_0/main/data_0_0_0?.., 
err: arrow/ipc: could not read message schema: arrow/ipc: could not read message metadata: unexpected EOF 

• setting application/json as response content-type

ERRO[0116]log.go:178 gosnowflake.(*defaultLogger).Errorf failed to extract HTTP response body. 
URL: https://my-customer-stage.s3.eu-central-1.amazonaws.com/stageid/results/01b0cef8-0102-25e0-0002-4776000d7012_0/main/data_0_0_0?.., 
err: gzip: invalid header 

• manually adding some text to the start of the chunk

ERRO[0251]log.go:178 gosnowflake.(*defaultLogger).Errorf failed to extract HTTP response body. 
URL: https://my-customer-stage.s3.eu-central-1.amazonaws.com/stageid/results/01b0cef8-0102-25e0-0002-4776000d7012_0/main/data_0_0_0?.. 
err: arrow/ipc: could not read message schema: arrow/ipc: could not read message metadata: unexpected EOF 

in all of the scenarios, the driver immediately retried the chunk download instead of panicking.

[till-kolditz]

Hi thanks for checking!
Unfortunately, we didn't log the payload, so not sure what exactly made buildFirstArrowChunk fail :-(
Let me try to create a go unit test for this.

[till-kolditz]

btw I already merged a fix in our forked repo: observeinc#17

[sfc-gh-dszmolka]

this is very helpful, thank you so much for pointing it out !

[sfc-gh-dszmolka]

fix coming under #993 . thank you so much for your contribution !

[sfc-gh-dszmolka]

and it's already merged into main. since we just had a release hours ago; it will be part of January's release. again the contribution and the pointers are very much appreciated !

[till-kolditz]

Great, I'm happy I could help out! And thanks for adding the unit test :-)

[davidhcoe]

apache/arrow-adbc#1454 is exhibiting the same arrow/ipc: could not read message schema: arrow/ipc: could not read message metadata: unexpected EOF message. Is this a related issue?

[sfc-gh-dszmolka]

released with gosnowflake 1.7.2