NoMethod error raised from redis instrumentation when queueing ActiveJob sidekiq job

[fcheung]

Description

When enqueueing a job I get an error with this backtrace

NoMethodError
undefined method `db' for #<RedisClient redis://127.0.0.1:6379/0>

      with_tracing(operation, statement: statement, database: client.db) { yield }

newrelic_rpm (8.13.0) lib/new_relic/agent/instrumentation/redis/instrumentation.rb in call_pipelined_with_tracing at line 35
newrelic_rpm (8.13.0) lib/new_relic/agent/instrumentation/redis/middleware.rb in call_pipelined at line 12
redis-client (0.11.1) lib/redis_client.rb in connect at line 677
redis-client (0.11.1) lib/redis_client.rb in raw_connection at line 647
redis-client (0.11.1) lib/redis_client.rb in ensure_connected at line 614
redis-client (0.11.1) lib/redis_client.rb in pipelined at line 349
redis-client (0.11.1) lib/redis_client/decorator.rb in pipelined at line 51
sidekiq (7.0.1) lib/sidekiq/client.rb in block in raw_push at line 214
connection_pool (2.3.0) lib/connection_pool.rb in block (2 levels) in with at line 65
connection_pool (2.3.0) lib/connection_pool.rb in handle_interrupt at line 64
connection_pool (2.3.0) lib/connection_pool.rb in block in with at line 64
connection_pool (2.3.0) lib/connection_pool.rb in handle_interrupt at line 61
connection_pool (2.3.0) lib/connection_pool.rb in with at line 61
sidekiq (7.0.1) lib/sidekiq/client.rb in raw_push at line 211
sidekiq (7.0.1) lib/sidekiq/client.rb in push at line 92
sidekiq (7.0.1) lib/sidekiq/client.rb in push at line 160
activejob (7.0.4) lib/active_job/queue_adapters/sidekiq_adapter.rb in enqueue at line 22
activejob (7.0.4) lib/active_job/enqueuing.rb in block in enqueue at line 67
activesupport (7.0.4) lib/active_support/callbacks.rb in block in run_callbacks at line 118
newrelic_rpm (8.13.0) lib/new_relic/agent/instrumentation/active_job.rb in block in run_in_trace at line 63
newrelic_rpm (8.13.0) lib/new_relic/agent/method_tracer.rb in block in trace_execution_scoped at line 74
newrelic_rpm (8.13.0) lib/new_relic/agent/method_tracer_helpers.rb in block in trace_execution_scoped at line 41
newrelic_rpm (8.13.0) lib/new_relic/agent/tracer.rb in capture_segment_error at line 355
newrelic_rpm (8.13.0) lib/new_relic/agent/method_tracer_helpers.rb in trace_execution_scoped at line 41
newrelic_rpm (8.13.0) lib/new_relic/agent/method_tracer.rb in trace_execution_scoped at line 72
newrelic_rpm (8.13.0) lib/new_relic/agent/instrumentation/active_job.rb in run_in_trace at line 61
newrelic_rpm (8.13.0) lib/new_relic/agent/instrumentation/active_job.rb in enqueue at line 38
newrelic_rpm (8.13.0) lib/new_relic/agent/instrumentation/active_job.rb in block (4 levels) in <main> at line 19
activesupport (7.0.4) lib/active_support/callbacks.rb in instance_exec at line 127
activesupport (7.0.4) lib/active_support/callbacks.rb in block in run_callbacks at line 127
activejob (7.0.4) lib/active_job/instrumentation.rb in block in instrument at line 25
activesupport (7.0.4) lib/active_support/notifications.rb in block in instrument at line 206
activesupport (7.0.4) lib/active_support/notifications/instrumenter.rb in instrument at line 24
activesupport (7.0.4) lib/active_support/notifications.rb in instrument at line 206
activejob (7.0.4) lib/active_job/instrumentation.rb in instrument at line 35
activejob (7.0.4) lib/active_job/instrumentation.rb in block (2 levels) in <module:Instrumentation> at line 9
activesupport (7.0.4) lib/active_support/callbacks.rb in instance_exec at line 127
activesupport (7.0.4) lib/active_support/callbacks.rb in block in run_callbacks at line 127
activesupport (7.0.4) lib/active_support/tagged_logging.rb in block in tagged at line 99
activesupport (7.0.4) lib/active_support/tagged_logging.rb in tagged at line 37
activesupport (7.0.4) lib/active_support/tagged_logging.rb in tagged at line 99
activejob (7.0.4) lib/active_job/logging.rb in tag_logger at line 25
activejob (7.0.4) lib/active_job/logging.rb in block (2 levels) in <module:Logging> at line 14
activesupport (7.0.4) lib/active_support/callbacks.rb in instance_exec at line 127
activesupport (7.0.4) lib/active_support/callbacks.rb in block in run_callbacks at line 127
activesupport (7.0.4) lib/active_support/callbacks.rb in run_callbacks at line 138
activejob (7.0.4) lib/active_job/enqueuing.rb in enqueue at line 63
activejob (7.0.4) lib/active_job/configured_job.rb in perform_later at line 15
actionmailer (7.0.4) lib/action_mailer/parameterized.rb in enqueue_delivery at line 144
actionmailer (7.0.4) lib/action_mailer/message_delivery.rb in deliver_later at line 99

Expected Behavior

No exception should be raised

Steps to Reproduce

This script reproduces the issue for me. requires REDIS_URL to be set ( eg redis://127.0.0.1:6379/0

require 'bundler/inline'

gemfile do
  source 'https://rubygems.org'
  gem 'rails', '~> 7.0.4'
  gem 'newrelic_rpm', '8.13'
  gem 'sidekiq', '7.0.1'
  gem 'redis', '4.8.0'
  gem 'redis-client', '0.11.1'
end

require 'active_job/railtie'
require 'action_mailer/railtie'
require 'logger'

Sidekiq.configure_server do |config|
  config.redis = { url: ENV["REDIS_URL"], id: nil }
end

Rails.logger = Logger.new(STDOUT)

class App < ::Rails::Application
  config.active_job.queue_adapter     = :sidekiq
  config.eager_load = false
end

class SomeMailer < ActionMailer::Base
  def message
    mail to: 'example@example.com', body:'test'
  end
end

App.initialize!
SomeMailer.message.deliver_later

Your Environment

ruby 3.1.2

Gem versions:

• rails 7.0.4
• newrelic_rpm 8.13
• sidekiq 7.0.1
• redis 5.0.5
• redis-client 0.11.1

Redis is the redis:6.2-alpine docker image, running locally. Everything works fine if I downgrade to newrelic 8.12, or downgrade the redis gem to 4.8

For Maintainers Only or Hero Triaging this bug

Suggested Priority (P1,P2,P3,P4,P5):
Suggested T-Shirt size (S, M, L, XL, Unknown):

[workato-integration]

https://issues.newrelic.com/browse/NEWRELIC-5367

[kaylareopelle]

Hi @fcheung, thanks for bringing this to our attention. We're taking a look at the problem and hope to have more info for you soon.

[kaylareopelle]

Hi @fcheung, we're triaging this bug. Thanks for bringing it to our attention and providing the reproduction!

To confirm, is this error crashing your application or your agent?

While we're working on a solution, we wanted to share some options with you to prevent the error until we find a solution:

• Instrumentation for Redis can be disabled in the configuration by setting instrumentation.redis to disabled. This should prevent the code raising the error from being hit
• If you want to be extra careful, you could also disable the instrumentation for Sidekiq too, by setting disable_sidekiq to true
• Downgrade the agent to 8.12.0 until a solution is found

cc: @stevenou

[fcheung]

It’s an application crash - anything that enqueues an activejob, for example trying to send an email. I’ve rolled back to 8.12 for now

[kaylareopelle]

Thanks for clarifying, @fcheung!

[kaylareopelle]

@fcheung @stevenou - We have a PR open with a potential fix for this bug. This can be tested by replacing your Gemfile entry for newrelic_rpm with the following:

gem 'newrelic_rpm', git: 'https://github.com/newrelic/newrelic-ruby-agent.git', branch: 'bugfix/redisclient_undefined_method_db'

If you choose to test this out, we'd love to hear how it goes.

[stevenou]

I only have this problem in dev, not production, so my testing is limited - but the PR appears to resolve the issue.

[fcheung]

Looks good in my dev environment.

[hannahramadan]

Hi @fcheung and @stevenou. Thank you both again for working with us to get this bug resolved. We just released v8.13.1 of the Ruby agent, which wraps up this issue. Cheers!