Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Pool hags after db disconnection #151

Open
CaioMF opened this issue Jan 12, 2022 · 1 comment
Open

Pool hags after db disconnection #151

CaioMF opened this issue Jan 12, 2022 · 1 comment

Comments

@CaioMF
Copy link

CaioMF commented Jan 12, 2022 •
edited
Loading

Bug Report

I have some APIs that when it is under low load, and the bank disconnects the connection, some connection validate failed errors occur

Versions

  • Driver: dev.miku:r2dbc-mysql:0.8.2.RELEASE
  • Database: MySQL
  • Java:11

Current Behavior

I have some APIs that when it is under low load, and the db disconnects the connection, some connection validate failed errors occur, soon the connection recovers and the error stops happening

Stack trace 
org.springframework.dao.DataAccessResourceFailureException: Failed to obtain R2DBC Connection; nested exception is io.r2dbc.spi.R2dbcNonTransientResourceException: Connection validation failed
	at org.springframework.r2dbc.connection.ConnectionFactoryUtils.lambda$getConnection$0(ConnectionFactoryUtils.java:88) ~[spring-r2dbc-5.3.8.jar!/:5.3.8]
	at reactor.core.publisher.Mono.lambda$onErrorMap$31(Mono.java:3488) ~[reactor-core-3.4.7.jar!/:3.4.7]
	at reactor.core.publisher.FluxOnErrorResume$ResumeSubscriber.onError(FluxOnErrorResume.java:94) ~[reactor-core-3.4.7.jar!/:3.4.7]
	at com.nr.instrumentation.reactor.netty.TokenLinkingSubscriber.lambda$onError$2(TokenLinkingSubscriber.java:46) ~[newrelic.jar:na]
	at com.nr.instrumentation.reactor.netty.TokenLinkingSubscriber.withNRError(TokenLinkingSubscriber.java:75) ~[newrelic.jar:na]
	at com.nr.instrumentation.reactor.netty.TokenLinkingSubscriber.onError(TokenLinkingSubscriber.java:46) ~[newrelic.jar:na]
	at reactor.core.publisher.FluxOnErrorResume$ResumeSubscriber.onError(FluxOnErrorResume.java:106) ~[reactor-core-3.4.7.jar!/:3.4.7]
	at reactor.core.publisher.FluxRetry$RetrySubscriber.onError(FluxRetry.java:94) ~[reactor-core-3.4.7.jar!/:3.4.7]
	at com.nr.instrumentation.reactor.netty.TokenLinkingSubscriber.lambda$onError$2(TokenLinkingSubscriber.java:46) ~[newrelic.jar:na]
	at com.nr.instrumentation.reactor.netty.TokenLinkingSubscriber.withNRError(TokenLinkingSubscriber.java:75) ~[newrelic.jar:na]
	at com.nr.instrumentation.reactor.netty.TokenLinkingSubscriber.onError(TokenLinkingSubscriber.java:46) ~[newrelic.jar:na]
	at reactor.core.publisher.FluxPeekFuseable$PeekFuseableSubscriber.onError(FluxPeekFuseable.java:234) ~[reactor-core-3.4.7.jar!/:3.4.7]
	at com.nr.instrumentation.reactor.netty.TokenLinkingSubscriber.lambda$onError$2(TokenLinkingSubscriber.java:46) ~[newrelic.jar:na]
	at com.nr.instrumentation.reactor.netty.TokenLinkingSubscriber.withNRError(TokenLinkingSubscriber.java:75) ~[newrelic.jar:na]
	at com.nr.instrumentation.reactor.netty.TokenLinkingSubscriber.onError(TokenLinkingSubscriber.java:46) ~[newrelic.jar:na]
	at reactor.core.publisher.FluxHide$SuppressFuseableSubscriber.onError(FluxHide.java:141) ~[reactor-core-3.4.7.jar!/:3.4.7]
	at reactor.core.publisher.MonoFlatMap$FlatMapMain.secondError(MonoFlatMap.java:192) ~[reactor-core-3.4.7.jar!/:3.4.7]
	at reactor.core.publisher.MonoFlatMap$FlatMapInner.onError(MonoFlatMap.java:259) ~[reactor-core-3.4.7.jar!/:3.4.7]
	at com.nr.instrumentation.reactor.netty.TokenLinkingSubscriber.lambda$onError$2(TokenLinkingSubscriber.java:46) ~[newrelic.jar:na]
	at com.nr.instrumentation.reactor.netty.TokenLinkingSubscriber.withNRError(TokenLinkingSubscriber.java:75) ~[newrelic.jar:na]
	at com.nr.instrumentation.reactor.netty.TokenLinkingSubscriber.onError(TokenLinkingSubscriber.java:46) ~[newrelic.jar:na]
	at reactor.core.publisher.FluxOnErrorResume$ResumeSubscriber.onError(FluxOnErrorResume.java:106) ~[reactor-core-3.4.7.jar!/:3.4.7]
	at com.nr.instrumentation.reactor.netty.TokenLinkingSubscriber.lambda$onError$2(TokenLinkingSubscriber.java:46) ~[newrelic.jar:na]
	at com.nr.instrumentation.reactor.netty.TokenLinkingSubscriber.withNRError(TokenLinkingSubscriber.java:75) ~[newrelic.jar:na]
	at com.nr.instrumentation.reactor.netty.TokenLinkingSubscriber.onError(TokenLinkingSubscriber.java:46) ~[newrelic.jar:na]
	at reactor.core.publisher.MonoIgnoreThen$ThenIgnoreMain.onError(MonoIgnoreThen.java:270) ~[reactor-core-3.4.7.jar!/:3.4.7]
	at reactor.core.publisher.Operators.error(Operators.java:197) ~[reactor-core-3.4.7.jar!/:3.4.7]
	at reactor.core.publisher.MonoError.subscribe(MonoError.java:52) ~[reactor-core-3.4.7.jar!/:3.4.7]
	at reactor.core.publisher.InternalMonoOperator.subscribe(InternalMonoOperator.java:64) ~[reactor-core-3.4.7.jar!/:3.4.7]
	at reactor.core.publisher.MonoIgnoreThen$ThenIgnoreMain.subscribeNext(MonoIgnoreThen.java:236) ~[reactor-core-3.4.7.jar!/:3.4.7]
	at reactor.core.publisher.MonoIgnoreThen$ThenIgnoreMain.onComplete(MonoIgnoreThen.java:203) ~[reactor-core-3.4.7.jar!/:3.4.7]
	at com.nr.instrumentation.reactor.netty.TokenLinkingSubscriber.withNRToken(TokenLinkingSubscriber.java:64) ~[newrelic.jar:na]
	at com.nr.instrumentation.reactor.netty.TokenLinkingSubscriber.onComplete(TokenLinkingSubscriber.java:51) ~[newrelic.jar:na]
	at com.nr.instrumentation.reactor.netty.TokenLinkingSubscriber.withNRToken(TokenLinkingSubscriber.java:64) ~[newrelic.jar:na]
	at com.nr.instrumentation.reactor.netty.TokenLinkingSubscriber.onComplete(TokenLinkingSubscriber.java:51) ~[newrelic.jar:na]
	at reactor.core.publisher.FluxDoFinally$DoFinallySubscriber.onComplete(FluxDoFinally.java:145) ~[reactor-core-3.4.7.jar!/:3.4.7]
	at com.nr.instrumentation.reactor.netty.TokenLinkingSubscriber.withNRToken(TokenLinkingSubscriber.java:64) ~[newrelic.jar:na]
	at com.nr.instrumentation.reactor.netty.TokenLinkingSubscriber.onComplete(TokenLinkingSubscriber.java:51) ~[newrelic.jar:na]
	at reactor.core.publisher.FluxHide$SuppressFuseableSubscriber.onComplete(FluxHide.java:146) ~[reactor-core-3.4.7.jar!/:3.4.7]
	at reactor.core.publisher.FluxDoFinally$DoFinallySubscriber.onComplete(FluxDoFinally.java:145) ~[reactor-core-3.4.7.jar!/:3.4.7]
	at com.nr.instrumentation.reactor.netty.TokenLinkingSubscriber.withNRToken(TokenLinkingSubscriber.java:64) ~[newrelic.jar:na]
	at com.nr.instrumentation.reactor.netty.TokenLinkingSubscriber.onComplete(TokenLinkingSubscriber.java:51) ~[newrelic.jar:na]
	at reactor.core.publisher.FluxHide$SuppressFuseableSubscriber.onComplete(FluxHide.java:146) ~[reactor-core-3.4.7.jar!/:3.4.7]
	at reactor.core.publisher.MonoPeekTerminal$MonoTerminalPeekSubscriber.onComplete(MonoPeekTerminal.java:299) ~[reactor-core-3.4.7.jar!/:3.4.7]
	at com.nr.instrumentation.reactor.netty.TokenLinkingSubscriber.withNRToken(TokenLinkingSubscriber.java:64) ~[newrelic.jar:na]
	at com.nr.instrumentation.reactor.netty.TokenLinkingSubscriber.onComplete(TokenLinkingSubscriber.java:51) ~[newrelic.jar:na]
	at reactor.core.publisher.FluxPeek$PeekSubscriber.onComplete(FluxPeek.java:259) ~[reactor-core-3.4.7.jar!/:3.4.7]
	at com.nr.instrumentation.reactor.netty.TokenLinkingSubscriber.withNRToken(TokenLinkingSubscriber.java:64) ~[newrelic.jar:na]
	at com.nr.instrumentation.reactor.netty.TokenLinkingSubscriber.onComplete(TokenLinkingSubscriber.java:51) ~[newrelic.jar:na]
	at reactor.core.publisher.MonoIgnoreThen$ThenIgnoreMain.onComplete(MonoIgnoreThen.java:209) ~[reactor-core-3.4.7.jar!/:3.4.7]
	at reactor.core.publisher.Operators.complete(Operators.java:136) ~[reactor-core-3.4.7.jar!/:3.4.7]
	at reactor.netty.FutureMono$DeferredFutureMono.subscribe(FutureMono.java:131) ~[reactor-netty-core-1.0.8.jar!/:1.0.8]
	at reactor.core.publisher.MonoIgnoreThen$ThenIgnoreMain.subscribeNext(MonoIgnoreThen.java:236) ~[reactor-core-3.4.7.jar!/:3.4.7]
	at reactor.core.publisher.MonoIgnoreThen$ThenIgnoreMain.onComplete(MonoIgnoreThen.java:203) ~[reactor-core-3.4.7.jar!/:3.4.7]
	at com.nr.instrumentation.reactor.netty.TokenLinkingSubscriber.withNRToken(TokenLinkingSubscriber.java:64) ~[newrelic.jar:na]
	at com.nr.instrumentation.reactor.netty.TokenLinkingSubscriber.onComplete(TokenLinkingSubscriber.java:51) ~[newrelic.jar:na]
	at reactor.core.publisher.Operators$MultiSubscriptionSubscriber.onComplete(Operators.java:2057) ~[reactor-core-3.4.7.jar!/:3.4.7]
	at com.nr.instrumentation.reactor.netty.TokenLinkingSubscriber.withNRToken(TokenLinkingSubscriber.java:64) ~[newrelic.jar:na]
	at com.nr.instrumentation.reactor.netty.TokenLinkingSubscriber.onComplete(TokenLinkingSubscriber.java:51) ~[newrelic.jar:na]
	at reactor.core.publisher.MonoFlatMap$FlatMapMain.onComplete(MonoFlatMap.java:181) ~[reactor-core-3.4.7.jar!/:3.4.7]
	at com.nr.instrumentation.reactor.netty.TokenLinkingSubscriber.withNRToken(TokenLinkingSubscriber.java:64) ~[newrelic.jar:na]
	at com.nr.instrumentation.reactor.netty.TokenLinkingSubscriber.onComplete(TokenLinkingSubscriber.java:51) ~[newrelic.jar:na]
	at reactor.core.publisher.MonoCreate$DefaultMonoSink.success(MonoCreate.java:133) ~[reactor-core-3.4.7.jar!/:3.4.7]
	at dev.miku.r2dbc.mysql.client.ReactorNettyClient.lambda$close$13(ReactorNettyClient.java:201) ~[r2dbc-mysql-0.8.2.RELEASE.jar!/:0.8.2.RELEASE]
	at reactor.core.publisher.MonoCreate.subscribe(MonoCreate.java:57) ~[reactor-core-3.4.7.jar!/:3.4.7]
	at reactor.core.publisher.Mono.subscribe(Mono.java:4150) ~[reactor-core-3.4.7.jar!/:3.4.7]
	at reactor.core.publisher.MonoIgnoreThen$ThenIgnoreMain.subscribeNext(MonoIgnoreThen.java:255) ~[reactor-core-3.4.7.jar!/:3.4.7]
	at reactor.core.publisher.MonoIgnoreThen.subscribe(MonoIgnoreThen.java:51) ~[reactor-core-3.4.7.jar!/:3.4.7]
	at reactor.core.publisher.InternalMonoOperator.subscribe(InternalMonoOperator.java:64) ~[reactor-core-3.4.7.jar!/:3.4.7]
	at reactor.core.publisher.MonoDefer.subscribe(MonoDefer.java:52) ~[reactor-core-3.4.7.jar!/:3.4.7]
	at reactor.core.publisher.Mono.subscribe(Mono.java:4150) ~[reactor-core-3.4.7.jar!/:3.4.7]
	at reactor.core.publisher.MonoIgnoreThen$ThenIgnoreMain.subscribeNext(MonoIgnoreThen.java:255) ~[reactor-core-3.4.7.jar!/:3.4.7]
	at reactor.core.publisher.MonoIgnoreThen.subscribe(MonoIgnoreThen.java:51) ~[reactor-core-3.4.7.jar!/:3.4.7]
	at reactor.core.publisher.Mono.subscribe(Mono.java:4150) ~[reactor-core-3.4.7.jar!/:3.4.7]
	at reactor.core.publisher.FluxOnErrorResume$ResumeSubscriber.onError(FluxOnErrorResume.java:103) ~[reactor-core-3.4.7.jar!/:3.4.7]
	at com.nr.instrumentation.reactor.netty.TokenLinkingSubscriber.lambda$onError$2(TokenLinkingSubscriber.java:46) ~[newrelic.jar:na]
	at com.nr.instrumentation.reactor.netty.TokenLinkingSubscriber.withNRError(TokenLinkingSubscriber.java:75) ~[newrelic.jar:na]
	at com.nr.instrumentation.reactor.netty.TokenLinkingSubscriber.onError(TokenLinkingSubscriber.java:46) ~[newrelic.jar:na]
	at reactor.core.publisher.MonoIgnoreThen$ThenIgnoreMain.onError(MonoIgnoreThen.java:270) ~[reactor-core-3.4.7.jar!/:3.4.7]
	at com.nr.instrumentation.reactor.netty.TokenLinkingSubscriber.lambda$onError$2(TokenLinkingSubscriber.java:46) ~[newrelic.jar:na]
	at com.nr.instrumentation.reactor.netty.TokenLinkingSubscriber.withNRError(TokenLinkingSubscriber.java:75) ~[newrelic.jar:na]
	at com.nr.instrumentation.reactor.netty.TokenLinkingSubscriber.onError(TokenLinkingSubscriber.java:46) ~[newrelic.jar:na]
	at reactor.core.publisher.MonoIgnoreElements$IgnoreElementsSubscriber.onError(MonoIgnoreElements.java:83) ~[reactor-core-3.4.7.jar!/:3.4.7]
	at com.nr.instrumentation.reactor.netty.TokenLinkingSubscriber.lambda$onError$2(TokenLinkingSubscriber.java:46) ~[newrelic.jar:na]
	at com.nr.instrumentation.reactor.netty.TokenLinkingSubscriber.withNRError(TokenLinkingSubscriber.java:75) ~[newrelic.jar:na]
	at com.nr.instrumentation.reactor.netty.TokenLinkingSubscriber.onError(TokenLinkingSubscriber.java:46) ~[newrelic.jar:na]
	at reactor.core.publisher.FluxHandleFuseable$HandleFuseableSubscriber.onNext(FluxHandleFuseable.java:191) ~[reactor-core-3.4.7.jar!/:3.4.7]
	at com.nr.instrumentation.reactor.netty.TokenLinkingSubscriber.lambda$onNext$1(TokenLinkingSubscriber.java:41) ~[newrelic.jar:na]
	at com.nr.instrumentation.reactor.netty.TokenLinkingSubscriber.withNRToken(TokenLinkingSubscriber.java:64) ~[newrelic.jar:na]
	at com.nr.instrumentation.reactor.netty.TokenLinkingSubscriber.onNext(TokenLinkingSubscriber.java:41) ~[newrelic.jar:na]
	at reactor.core.publisher.FluxHide$SuppressFuseableSubscriber.onNext(FluxHide.java:136) ~[reactor-core-3.4.7.jar!/:3.4.7]
	at com.nr.instrumentation.reactor.netty.TokenLinkingSubscriber.lambda$onNext$1(TokenLinkingSubscriber.java:41) ~[newrelic.jar:na]
	at com.nr.instrumentation.reactor.netty.TokenLinkingSubscriber.withNRToken(TokenLinkingSubscriber.java:64) ~[newrelic.jar:na]
	at com.nr.instrumentation.reactor.netty.TokenLinkingSubscriber.onNext(TokenLinkingSubscriber.java:41) ~[newrelic.jar:na]
	at reactor.core.publisher.FluxHide$SuppressFuseableSubscriber.onNext(FluxHide.java:136) ~[reactor-core-3.4.7.jar!/:3.4.7]
	at reactor.core.publisher.Operators$MonoSubscriber.complete(Operators.java:1815) ~[reactor-core-3.4.7.jar!/:3.4.7]
	at reactor.core.publisher.MonoSupplier.subscribe(MonoSupplier.java:61) ~[reactor-core-3.4.7.jar!/:3.4.7]
	at reactor.core.publisher.Mono.subscribe(Mono.java:4150) ~[reactor-core-3.4.7.jar!/:3.4.7]
	at reactor.core.publisher.MonoIgnoreThen$ThenIgnoreMain.subscribeNext(MonoIgnoreThen.java:255) ~[reactor-core-3.4.7.jar!/:3.4.7]
	at reactor.core.publisher.MonoIgnoreThen.subscribe(MonoIgnoreThen.java:51) ~[reactor-core-3.4.7.jar!/:3.4.7]
	at reactor.core.publisher.InternalMonoOperator.subscribe(InternalMonoOperator.java:64) ~[reactor-core-3.4.7.jar!/:3.4.7]
	at reactor.core.publisher.FluxHide$SuppressFuseableSubscriber.onNext(FluxHide.java:136) ~[reactor-core-3.4.7.jar!/:3.4.7]
	at reactor.core.publisher.Operators$MonoSubscriber.complete(Operators.java:1815) ~[reactor-core-3.4.7.jar!/:3.4.7]
	at reactor.core.publisher.MonoFlatMap$FlatMapInner.onNext(MonoFlatMap.java:249) ~[reactor-core-3.4.7.jar!/:3.4.7]
	at com.nr.instrumentation.reactor.netty.TokenLinkingSubscriber.lambda$onNext$1(TokenLinkingSubscriber.java:41) ~[newrelic.jar:na]
	at com.nr.instrumentation.reactor.netty.TokenLinkingSubscriber.withNRToken(TokenLinkingSubscriber.java:64) ~[newrelic.jar:na]
	at com.nr.instrumentation.reactor.netty.TokenLinkingSubscriber.onNext(TokenLinkingSubscriber.java:41) ~[newrelic.jar:na]
	at reactor.core.publisher.FluxOnErrorResume$ResumeSubscriber.onNext(FluxOnErrorResume.java:79) ~[reactor-core-3.4.7.jar!/:3.4.7]
	at reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.onNext(FluxOnAssembly.java:387) ~[reactor-core-3.4.7.jar!/:3.4.7]
	at com.nr.instrumentation.reactor.netty.TokenLinkingSubscriber.lambda$onNext$1(TokenLinkingSubscriber.java:41) ~[newrelic.jar:na]
	at com.nr.instrumentation.reactor.netty.TokenLinkingSubscriber.withNRToken(TokenLinkingSubscriber.java:64) ~[newrelic.jar:na]
	at com.nr.instrumentation.reactor.netty.TokenLinkingSubscriber.onNext(TokenLinkingSubscriber.java:41) ~[newrelic.jar:na]
	at reactor.core.publisher.FluxHide$SuppressFuseableSubscriber.onNext(FluxHide.java:136) ~[reactor-core-3.4.7.jar!/:3.4.7]
	at reactor.core.publisher.Operators$MonoSubscriber.complete(Operators.java:1815) ~[reactor-core-3.4.7.jar!/:3.4.7]
	at reactor.core.publisher.MonoFlatMap$FlatMapInner.onNext(MonoFlatMap.java:249) ~[reactor-core-3.4.7.jar!/:3.4.7]
	at reactor.core.publisher.Operators$ScalarSubscription.request(Operators.java:2397) ~[reactor-core-3.4.7.jar!/:3.4.7]
	at reactor.core.publisher.MonoFlatMap$FlatMapInner.onSubscribe(MonoFlatMap.java:238) ~[reactor-core-3.4.7.jar!/:3.4.7]
	at reactor.core.publisher.MonoJust.subscribe(MonoJust.java:54) ~[reactor-core-3.4.7.jar!/:3.4.7]
	at reactor.core.publisher.InternalMonoOperator.subscribe(InternalMonoOperator.java:64) ~[reactor-core-3.4.7.jar!/:3.4.7]
	at reactor.core.publisher.MonoFlatMap$FlatMapMain.onNext(MonoFlatMap.java:157) ~[reactor-core-3.4.7.jar!/:3.4.7]
	at reactor.core.publisher.FluxContextWrite$ContextWriteSubscriber.onNext(FluxContextWrite.java:107) ~[reactor-core-3.4.7.jar!/:3.4.7]
	at com.nr.instrumentation.reactor.netty.TokenLinkingSubscriber.lambda$onNext$1(TokenLinkingSubscriber.java:41) ~[newrelic.jar:na]
	at com.nr.instrumentation.reactor.netty.TokenLinkingSubscriber.withNRToken(TokenLinkingSubscriber.java:64) ~[newrelic.jar:na]
	at com.nr.instrumentation.reactor.netty.TokenLinkingSubscriber.onNext(TokenLinkingSubscriber.java:41) ~[newrelic.jar:na]
	at reactor.core.publisher.FluxHide$SuppressFuseableSubscriber.onNext(FluxHide.java:136) ~[reactor-core-3.4.7.jar!/:3.4.7]
	at reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.onNext(FluxMapFuseable.java:127) ~[reactor-core-3.4.7.jar!/:3.4.7]
	at com.nr.instrumentation.reactor.netty.TokenLinkingSubscriber.lambda$onNext$1(TokenLinkingSubscriber.java:41) ~[newrelic.jar:na]
	at reactor.core.publisher.Operators$MonoSubscriber.complete(Operators.java:1815) ~[reactor-core-3.4.7.jar!/:3.4.7]
	at reactor.core.publisher.MonoCollect$CollectSubscriber.onComplete(MonoCollect.java:159) ~[reactor-core-3.4.7.jar!/:3.4.7]
	at com.nr.instrumentation.reactor.netty.TokenLinkingSubscriber.withNRToken(TokenLinkingSubscriber.java:64) ~[newrelic.jar:na]
	at com.nr.instrumentation.reactor.netty.TokenLinkingSubscriber.onComplete(TokenLinkingSubscriber.java:51) ~[newrelic.jar:na]
	at reactor.core.publisher.FluxMap$MapSubscriber.onComplete(FluxMap.java:142) ~[reactor-core-3.4.7.jar!/:3.4.7]
	at com.nr.instrumentation.reactor.netty.TokenLinkingSubscriber.withNRToken(TokenLinkingSubscriber.java:64) ~[newrelic.jar:na]
	at com.nr.instrumentation.reactor.netty.TokenLinkingSubscriber.onComplete(TokenLinkingSubscriber.java:51) ~[newrelic.jar:na]
	at reactor.core.publisher.FluxPeek$PeekSubscriber.onComplete(FluxPeek.java:259) ~[reactor-core-3.4.7.jar!/:3.4.7]
	at com.nr.instrumentation.reactor.netty.TokenLinkingSubscriber.withNRToken(TokenLinkingSubscriber.java:64) ~[newrelic.jar:na]
	at com.nr.instrumentation.reactor.netty.TokenLinkingSubscriber.onComplete(TokenLinkingSubscriber.java:51) ~[newrelic.jar:na]
	at reactor.core.publisher.FluxMap$MapSubscriber.onComplete(FluxMap.java:142) ~[reactor-core-3.4.7.jar!/:3.4.7]
	at reactor.netty.channel.FluxReceive.onInboundComplete(FluxReceive.java:401) ~[reactor-netty-core-1.0.8.jar!/:1.0.8]
	at reactor.netty.channel.ChannelOperations.onInboundComplete(ChannelOperations.java:420) ~[reactor-netty-core-1.0.8.jar!/:1.0.8]
	at reactor.netty.channel.ChannelOperations.terminate(ChannelOperations.java:474) ~[reactor-netty-core-1.0.8.jar!/:1.0.8]
	at reactor.netty.http.client.HttpClientOperations.onInboundNext(HttpClientOperations.java:685) ~[reactor-netty-http-1.0.8.jar!/:1.0.8]
	at reactor.netty.channel.ChannelOperationsHandler.channelRead(ChannelOperationsHandler.java:94) ~[reactor-netty-core-1.0.8.jar!/:1.0.8]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[netty-transport-4.1.65.Final.jar!/:4.1.65.Final]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[netty-transport-4.1.65.Final.jar!/:4.1.65.Final]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[netty-transport-4.1.65.Final.jar!/:4.1.65.Final]
	at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) ~[netty-codec-4.1.65.Final.jar!/:4.1.65.Final]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[netty-transport-4.1.65.Final.jar!/:4.1.65.Final]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[netty-transport-4.1.65.Final.jar!/:4.1.65.Final]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[netty-transport-4.1.65.Final.jar!/:4.1.65.Final]
	at io.netty.channel.CombinedChannelDuplexHandler$DelegatingChannelHandlerContext.fireChannelRead(CombinedChannelDuplexHandler.java:436) ~[netty-transport-4.1.65.Final.jar!/:4.1.65.Final]
	at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) ~[netty-codec-4.1.65.Final.jar!/:4.1.65.Final]
	at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) ~[netty-codec-4.1.65.Final.jar!/:4.1.65.Final]
	at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) ~[netty-transport-native-epoll-4.1.65.Final-linux-x86_64.jar!/:4.1.65.Final]
	at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) ~[netty-transport-native-epoll-4.1.65.Final-linux-x86_64.jar!/:4.1.65.Final]
	at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) ~[netty-transport-native-epoll-4.1.65.Final-linux-x86_64.jar!/:4.1.65.Final]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[netty-common-4.1.65.Final.jar!/:4.1.65.Final]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[netty-common-4.1.65.Final.jar!/:4.1.65.Final]
	at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[netty-common-4.1.65.Final.jar!/:4.1.65.Final]
	at java.base/java.lang.Thread.run(Thread.java:829) ~[na:na]
Caused by: io.r2dbc.spi.R2dbcNonTransientResourceException: Connection validation failed
	at io.r2dbc.pool.Validation.lambda$validate$2(Validation.java:45) ~[r2dbc-pool-0.8.7.RELEASE.jar!/:0.8.7.RELEASE]
	at reactor.core.publisher.FluxHandleFuseable$HandleFuseableSubscriber.onNext(FluxHandleFuseable.java:169) ~[reactor-core-3.4.7.jar!/:3.4.7]
	... 171 common frames omitted

Table schema

Input Code 
-- your SQL here;

Steps to reproduce

Input Code 
// your code here;

Expected behavior/code

Possible Solution

Additional context

I've already tried in the following settings and in all the behavior persists, and the wait_timeout of my dbs are greater than the max_idle_time

spring:
  r2dbc:
    url: url
    username: user
    password: pass
    pool:
      max-idle-time: 10m

spring:
  r2dbc:
    url: url
    username: user
    password: pass
    pool:
      max-idle-time: 10m
      validation-depth: remote

I think it might be related to the issue 107
@zitniak
Copy link

zitniak commented Apr 11, 2024

I occasionally experience the same issue in the PROD.

Versions:

Driver: io.asyncer:r2dbc-mysql:1.1.2
Database: MySQL
Java:17

Stack trace 
org.springframework.dao.DataAccessResourceFailureException: Failed to obtain R2DBC Connection
	at org.springframework.r2dbc.connection.ConnectionFactoryUtils.lambda$getConnection$0(ConnectionFactoryUtils.java:100)
	at reactor.core.publisher.Mono.lambda$onErrorMap$28(Mono.java:3799)
	at reactor.core.publisher.FluxOnErrorResume$ResumeSubscriber.onError(FluxOnErrorResume.java:94)
	at reactor.core.publisher.FluxOnErrorResume$ResumeSubscriber.onError(FluxOnErrorResume.java:106)
	at reactor.core.publisher.FluxRetry$RetrySubscriber.onError(FluxRetry.java:96)
	at reactor.core.publisher.MonoFlatMap$FlatMapMain.secondError(MonoFlatMap.java:241)
	at reactor.core.publisher.MonoFlatMap$FlatMapInner.onError(MonoFlatMap.java:315)
	at io.r2dbc.pool.MonoDiscardOnCancel$MonoDiscardOnCancelSubscriber.onError(MonoDiscardOnCancel.java:98)
	at reactor.core.publisher.FluxOnErrorResume$ResumeSubscriber.onError(FluxOnErrorResume.java:106)
	at reactor.core.publisher.MonoIgnoreThen$ThenIgnoreMain.onError(MonoIgnoreThen.java:279)
	at reactor.core.publisher.MonoIgnoreThen$ThenIgnoreMain.subscribeNext(MonoIgnoreThen.java:232)
	at reactor.core.publisher.MonoIgnoreThen$ThenIgnoreMain.onComplete(MonoIgnoreThen.java:204)
	at reactor.core.publisher.FluxDoFinally$DoFinallySubscriber.onComplete(FluxDoFinally.java:128)
	at reactor.core.publisher.FluxDoFinally$DoFinallySubscriber.onComplete(FluxDoFinally.java:128)
	at reactor.core.publisher.MonoIgnoreThen$ThenIgnoreMain.onComplete(MonoIgnoreThen.java:210)
	at reactor.netty.FutureMono$FutureSubscription.operationComplete(FutureMono.java:196)
	at io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590)
	at io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:557)
	at io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492)
	at io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636)
	at io.netty.util.concurrent.DefaultPromise.setSuccess0(DefaultPromise.java:625)
	at io.netty.util.concurrent.DefaultPromise.trySuccess(DefaultPromise.java:105)
	at io.netty.channel.DefaultChannelPromise.trySuccess(DefaultChannelPromise.java:84)
	at io.netty.channel.AbstractChannel$AbstractUnsafe.safeSetSuccess(AbstractChannel.java:990)
	at io.netty.channel.AbstractChannel$AbstractUnsafe.close(AbstractChannel.java:686)
	at io.netty.channel.AbstractChannel$AbstractUnsafe.close(AbstractChannel.java:620)
	at io.netty.channel.DefaultChannelPipeline$HeadContext.close(DefaultChannelPipeline.java:1352)
	at io.netty.channel.AbstractChannelHandlerContext.invokeClose(AbstractChannelHandlerContext.java:749)
	at io.netty.channel.AbstractChannelHandlerContext.access$1200(AbstractChannelHandlerContext.java:61)
	at io.netty.channel.AbstractChannelHandlerContext$11.run(AbstractChannelHandlerContext.java:732)
	at io.netty.util.concurrent.AbstractEventExecutor.runTask(AbstractEventExecutor.java:173)
	at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:166)
	at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:470)
	at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:413)
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997)
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
	at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
	at java.base/java.lang.Thread.run(Thread.java:840)
Caused by: io.r2dbc.spi.R2dbcNonTransientResourceException: Connection validation failed
	at io.r2dbc.pool.Validation.lambda$validate$2(Validation.java:45)
	at reactor.core.publisher.FluxHandleFuseable$HandleFuseableSubscriber.onNext(FluxHandleFuseable.java:179)
	at reactor.core.publisher.MonoSupplier$MonoSupplierSubscription.request(MonoSupplier.java:145)
	at reactor.core.publisher.FluxHandleFuseable$HandleFuseableSubscriber.request(FluxHandleFuseable.java:260)
	at reactor.core.publisher.MonoIgnoreElements$IgnoreElementsSubscriber.onSubscribe(MonoIgnoreElements.java:72)
	at reactor.core.publisher.FluxHandleFuseable$HandleFuseableSubscriber.onSubscribe(FluxHandleFuseable.java:164)
	at reactor.core.publisher.MonoSupplier.subscribe(MonoSupplier.java:48)
	at reactor.core.publisher.InternalMonoOperator.subscribe(InternalMonoOperator.java:76)
	at reactor.core.publisher.MonoIgnoreThen$ThenIgnoreMain.subscribeNext(MonoIgnoreThen.java:264)
	at reactor.core.publisher.MonoIgnoreThen.subscribe(MonoIgnoreThen.java:51)
	at reactor.core.publisher.MonoIgnoreThen$ThenIgnoreMain.subscribeNext(MonoIgnoreThen.java:241)
	at reactor.core.publisher.MonoIgnoreThen.subscribe(MonoIgnoreThen.java:51)
	at reactor.core.publisher.InternalMonoOperator.subscribe(InternalMonoOperator.java:76)
	at io.r2dbc.pool.MonoDiscardOnCancel.subscribe(MonoDiscardOnCancel.java:50)
	at reactor.core.publisher.MonoFlatMap$FlatMapMain.onNext(MonoFlatMap.java:165)
	at reactor.pool.AbstractPool$Borrower.deliver(AbstractPool.java:480)
	at reactor.pool.SimpleDequePool.lambda$drainLoop$8(SimpleDequePool.java:372)
	at reactor.core.scheduler.ImmediateScheduler.schedule(ImmediateScheduler.java:52)
	at reactor.pool.SimpleDequePool.drainLoop(SimpleDequePool.java:372)
	at reactor.pool.SimpleDequePool.pendingOffer(SimpleDequePool.java:604)
	at reactor.pool.SimpleDequePool.doAcquire(SimpleDequePool.java:298)
	at reactor.pool.AbstractPool$Borrower.request(AbstractPool.java:436)
	at reactor.core.publisher.MonoFlatMap$FlatMapMain.request(MonoFlatMap.java:194)
	at reactor.core.publisher.Operators$MultiSubscriptionSubscriber.set(Operators.java:2367)
	at reactor.core.publisher.Operators$MultiSubscriptionSubscriber.onSubscribe(Operators.java:2241)
	at reactor.core.publisher.MonoFlatMap$FlatMapMain.onSubscribe(MonoFlatMap.java:117)
	at reactor.pool.SimpleDequePool$QueueBorrowerMono.subscribe(SimpleDequePool.java:722)
	at reactor.core.publisher.InternalMonoOperator.subscribe(InternalMonoOperator.java:76)
	at reactor.core.publisher.MonoDefer.subscribe(MonoDefer.java:53)
	at reactor.core.publisher.FluxRetry$RetrySubscriber.resubscribe(FluxRetry.java:120)
	at reactor.core.publisher.FluxRetry$RetrySubscriber.onError(FluxRetry.java:102)
	... 33 common frames omitted

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@zitniak @CaioMF