Closing the stream because it has been inactive for 600 seconds

[tarunrawat87]

Thanks for stopping by to ask us a question! Please make sure to include:

• What you're trying to do

I'm trying to write to BQ Table via storage API . I have used WriterClient to createSteamConnection and I append rows once the stream is created.

• What code you've already tried

const connection = await this.client.write.createStreamConnection({
                streamType,
                destinationTable
            });
            let instance = this;
            connection.onConnectionError((err)=>{
                console.log('err has happened',err)
                instance.init(req);
            })

            const streamId = connection.getStreamId();

            const writer = new Writer({
                streamId,
                connection,
                protoDescriptor

            });
then later on 

writer.appendRows({ serializedRows: rows });

`

• Any error messages you're getting
  code: 10,
  details: 'Closing the stream because it has been inactive for 600 seconds.

PLEASE READ: If you have a support contract with Google, please create an issue in the support console instead of filing on GitHub. This will ensure a timely response.

[alvarowolfx]

hey @tarunrawat87 thanks for the report. The managedwriter package handles reconnections, the BigQuery Write API proactively closes the connection after 10 minutes of inactivity, so you don't need to handle that case. Can you provide more information of what is happening on the instance.init call ? I wonder if this is creating more connectionErrorevent handles and maybe interfering with the mechanism for reconnections.

References:

• https://cloud.google.com/bigquery/docs/write-api#connections

BigQuery closes the gRPC connection if the connection remains idle for too long. If this happens, the response code is HTTP 409. The gRPC connection can also be closed in the event of a server restart or for other reasons. If a connection error occurs, create a new connection.

[tarunrawat87]

@alvarowolfx init function creates a new stream connection and create a new Writer instance using newly created streamId. Can you tell me if there is any configuration I can pass to increase the timeout of idle steam ?

[alvarowolfx]

I ran some tests here and indeed the reconnect functionality is broken because the service side changed a bit how the disconnection happen. I'll send a PR to fix this.

About this, this is not configurable, is a restriction on the service side to terminate client connections that are idle, as they are expensive to maintain and kept open. So it's not possible configure it via the library.

@alvarowolfx init function creates a new stream connection and create a new Writer instance using newly created streamId. Can you tell me if there is any configuration I can pass to increase the timeout of idle steam ?

[tarunrawat87]

@alvarowolfx
When this fix will be deployed and when you say "reconnect"
what do you mean by it ?
If steam get closed after 600 seconds , do I have do something "code wise" to reconnect or it is handled by library itself.
When I say "code wise" : I mean do I need to create new stream connection and Writer object in case of stream close or I don't ?

[alvarowolfx]

The reconnection is handled by the library itself, no need to create a new stream connection and writer.

@alvarowolfx When this fix will be deployed and when you say "reconnect" what do you mean by it ? If steam get closed after 600 seconds , do I have do something "code wise" to reconnect or it is handled by library itself. When I say "code wise" : I mean do I need to create new stream connection and Writer object in case of stream close or I don't ?