Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Upgrade Rails #2872

Closed
rsgonzal opened this issue Mar 19, 2021 · 17 comments
Closed

Upgrade Rails #2872

rsgonzal opened this issue Mar 19, 2021 · 17 comments
Assignees
Labels
Contractors Work that can be effectively handed off to dev support team ready

Comments

@rsgonzal
Copy link
Collaborator

rsgonzal commented Mar 19, 2021

Deliverables

  • Upgrade Rails to the latest version (6.1.3.1 when this ticket was written)
  • Document the changes that were needed to go from 5.2 to 6.0
  • Document the changes that were needed to go from 6.0 to 6.1
  • After upgrading Rails, replace sprockets with webpacker
  • Document the steps taken to replace sprockets with webpacker
  • Open a PR

Deployment Notes
Do not deploy

@rsgonzal rsgonzal added 0 - Icebox Contractors Work that can be effectively handed off to dev support team labels Mar 19, 2021
@tamtran-agilityio
Copy link
Contributor

@rsgonzal @viviancan I have update version Rails and some gem. When I update version Rails have many method helpers throw error on view and testing. I need more time to finish ticket, thanks!

@rsgonzal
Copy link
Collaborator Author

rsgonzal commented Jul 2, 2021

Thanks for the update! Please let us know if you have any questions.

@tamtran-agilityio
Copy link
Contributor

@rsgonzal @shaunxp20 @viviancan
Current on feature have used version rails 6.0.0
With verions rails 6.1.3.2 we can update Gemfile and remove Gemfile.lock and update content file Gemfile with list gem below

source "https://rubygems.org"
ruby “2.6.4”

gem "rails", "~> 6.1.3.2"

gem "puma", "~> 3.12.6"
gem "pg", "~> 1.2.3"
gem "casting", "~> 0.7"

gem "barnes", "~> 0.0.7"

gem "bootsnap", "~> 1.3", require: false

gem "fast_jsonapi", "~> 1.4"
gem "httparty", "~> 0.16"

gem "textacular", "~> 5.1"

gem "pdf-forms", "~> 1.2"

gem "public_activity", "~> 1.6"
gem "paranoia", "~> 2.4"
gem "counter_culture", "~> 1.12"

gem "autoprefixer-rails", "~> 6.7"
gem "uglifier", "~> 3.2"
gem "coffee-rails", "~> 5.0"

gem "turbolinks", "~> 5.2"
gem "jquery-rails", "~> 4.3"
gem "jquery-ui-rails", "~> 6.0"
gem "lodash-rails", "~> 4.17"
gem "dropzonejs-rails", "~> 0.8"
gem "webpacker", "~> 5.x"

gem "cocoon", "~> 1.2"

gem "sass-rails", "~> 5.0.7"
gem "normalize-rails", "~> 4.1"

gem "rails-i18n", "~> 6.0.0"
gem "i18n-tasks", "~> 0.9"
gem "clipboard-rails", "~> 1.7"
gem "will_paginate", "~> 3.1"

gem "datagrid", "~> 1.5"

gem "simple_form", "~> 5.0"
gem "country_state_select", "~> 3.0"

gem "nokogiri", "~> 1.11.0"

gem "friendly_id", "~> 5.2"

gem "bcrypt", "~> 3.1"

gem "sidekiq", "~> 4.2"

gem "airbrake", "~> 11.0.0"
gem "newrelic_rpm", "~> 7.1.0"
gem "scout_apm", "~> 2.4"

gem "dalli", "~> 2.7"

gem "geocoder", "~> 1.4.9"
gem "timezone", "~> 1.3"

gem "carrierwave", "~> 1.2"
gem "mini_magick", "4.9.4"
gem "fog-aws", "~> 1.4"
gem "carrierwave_direct", "~> 0.0"

gem "checkr-official", "~> 1.5", require: "checkr"

gem "countries", "~> 1.2", require: "countries/global"

gem "city-state",
  git: "https://github.com/Iridescent-CM/city-state.git",
  branch: "keep-list-updated"

gem "carmen", "~> 1.1"

gem "indefinite_article", "~> 0.2"

gem "rack-rewrite", "~> 1.5", require: "rack/rewrite"

gem "browser", "~> 2.5"

gem "loofah", "~> 2.2"

gem "dotenv-rails", "~> 2.5"

gem "pdfkit", "0.8.4.3.2"

gem "gibbon", "~> 3.3.0"

group :development do
  gem "pp_sql", "~> 0.2", require: false
  gem "active_record_query_trace", "~> 1.5"
  gem "letter_opener", "~> 1.6"
  gem "listen", "~> 3.1"
  gem "churn", "~> 1.0", require: false
  gem "web-console", "~> 3.7"
  gem "spring", "~> 2.0"
  gem "spring-watcher-listen", "~> 2.0"
  gem "mailgun-ruby", "~> 1.1"
end

group :test do
  gem "timecop", "~> 0.9"
  gem "vcr", "~> 3.0"
  gem "webmock", "~> 3.5"
  gem "sinatra", "~> 2.0"

  gem "rails-controller-testing", "~> 1.0"
  gem "fakeredis", "~> 0.7", require: "fakeredis/rspec"

  gem "capybara", "~> 3.31"
  gem "capybara-email", "~> 3.0.2"
  gem "webdrivers", "~> 4.0"
  gem "rspec-rails", "~> 3.8"

  gem "pdf-reader", "~> 2.4"

  # deprecated
  gem "font-awesome-rails", "~> 4.7"
end

group :development, :test do
  gem "standard"
end

group :production, :development do
  gem "premailer-rails", "~> 1.10"
end

group :production do
  gem "hiredis", "~> 0.6"
  gem "rack-timeout", "~> 0.5"
  gem "heroku-deflater", "~> 0.6"
  gem "rails_autoscale_agent"
  gem "cloudflare-rails", "~> 2.2.0"
  gem "wkhtmltopdf-heroku", "2.12.5.0"
end

gem "factory_bot_rails", "~> 4.11"

group :legacy do
  # LEGACY MIGRATION
  # Do not use these gems
  #
  gem "flag_shih_tzu", "~> 0.3", require: false
  gem "aws-sdk", "~> 2.11", require: false
end


shaun-technovation added a commit that referenced this issue Jul 29, 2021
Refs: #2872
shaun-technovation added a commit that referenced this issue Jul 29, 2021
Refs: #2872
shaun-technovation added a commit that referenced this issue Jul 29, 2021
Refs: #2872
shaun-technovation added a commit that referenced this issue Jul 29, 2021
Refs: #2872
shaun-technovation added a commit that referenced this issue Jul 29, 2021
Refs: #2872
shaun-technovation added a commit that referenced this issue Jul 30, 2021
Refs: #2872
shaun-technovation added a commit that referenced this issue Jul 30, 2021
Refs: #2872
shaun-technovation added a commit that referenced this issue Jul 30, 2021
Refs: #2872
shaun-technovation added a commit that referenced this issue Jul 31, 2021
Refs: #2872
shaun-technovation added a commit that referenced this issue Aug 2, 2021
From what I could tell, we're not even using CoffeeScript, so I think
it's safe to remove this. 👍

Refs: #2872
shaun-technovation added a commit that referenced this issue Aug 2, 2021
Updating this to RSpec 4.x, instead of 5.x, seems safer to make a
smaller jump first; after some other kinks get worked out, then we can
look at updating this to version 5.x. 👍

I also removed the Selenium web drivers, unless I'm missing something, I
don't think we need those at this point.

Refs: #2872
shaun-technovation added a commit that referenced this issue Aug 2, 2021
`config.ignore_localhost = true` - this seems like a valid change, but
at this point there are too many moving pieces to the Rails upgrade,
which I'm trying to minimize, so that's why I'm removing it

`config.allow_http_connections_when_no_cassette = true` - we definitely
don't want this setting enabled

`github-releases.githubusercontent.com` - this seems to be the new
domain where the Chrome/Gecko drivers get downloaded from, so we want to
allow connections to there

Refs: #2872
shaun-technovation added a commit that referenced this issue Aug 2, 2021
Just fixing some linting/formatting issues.

Refs: #2872
shaun-technovation added a commit that referenced this issue Aug 2, 2021
- Explicitly expect `true` and `false`, instead of truthy/falsey values
- Apparently "null" gets returned for empty survey values now 🤷
- Ensure the admin is fully signed in (by expecting the text "Welcome
  back!") before moving on to the next steps

Refs: #2872
shaun-technovation added a commit that referenced this issue Aug 2, 2021
"Judge" is a better name than "user".

“There are only two industries that call their customers ‘users’:
illegal drugs and software” - Edward Tufte

Refs: #2872
shaun-technovation added a commit that referenced this issue Aug 2, 2021
- Reverting the changes Agility made
- Marking them as pending; spending too much time on trying to
  fix them, need a break from them for now

Refs: #2872
shaun-technovation added a commit that referenced this issue Aug 2, 2021
- Explicitly expect `true` and `false`, instead of truthy/falsey values
- Be explicit about expecting empty strings
- Ensure the admin is fully signed in (by expecting the text "Welcome
  back!") before moving on to the next steps

Refs: #2872
shaun-technovation added a commit that referenced this issue Aug 2, 2021
"Judge" is a better name than "user".

“There are only two industries that call their customers ‘users’:
illegal drugs and software” - Edward Tufte

Refs: #2872
shaun-technovation added a commit that referenced this issue Aug 2, 2021
- Reverting the changes Agility made
- Marking them as pending; spending too much time on trying to
  fix them, need a break from them for now

Refs: #2872
shaun-technovation added a commit that referenced this issue Aug 2, 2021
shaun-technovation added a commit that referenced this issue Aug 3, 2021
These specs are failing intermittently, after some researching I think
this new way will work better, we'll see!

Some Stack Overflow answers for reference:
https://stackoverflow.com/a/60695813
https://stackoverflow.com/a/52405269
https://stackoverflow.com/a/44916498

Refs: #2872
shaun-technovation added a commit that referenced this issue Aug 3, 2021
Changing these back to their original form, i.e., removing the changes
from Agility.

Refs: #2872
shaun-technovation added a commit that referenced this issue Aug 7, 2021
Attempting to fix another flaky test.

Refs: #2872
shaun-technovation added a commit that referenced this issue Aug 7, 2021
Attempting to fix another flaky test.

Refs: #2872
@shaun-technovation
Copy link
Contributor

This is getting to a pretty good/stable spot. I think the next step is to get it on QA for some additional testing (I only did very basic manual testing).

Some random notes/comments:

  • two RSpec tests regarding Checkr have been temporarily disabled (I couldn't figure them out and I was spending too much time on them), we should manually QA the Checkr functionality to make sure it's still working okay; I also created Address "pending" RSpec tests #3020 to follow up with the tests
  • Rails was upgraded to the latest version 6.1, various other things were upgraded too:
    • Ruby 2.7.4
    • NodeJS 14
    • various other gems, like pg, sidekiq, sass-rails, and a bunch of testing libraries
  • the CircleCI build is passing more consistently now (more than 25 times in a row)

@shaun-technovation
Copy link
Contributor

Thank you for your help getting this one started, @tamtran-agilityio!! We appreciate it! 🙌 🎉 😀

@shaun-technovation
Copy link
Contributor

I also forgot to mention, I created a new config setting for the mentor training doc MENTOR_TRAINING_DOC_URL, I already set this up on Heroku for both QA and PROD to the existing doc: https://docs.google.com/presentation/d/e/2PACX- 1vSN6IcqYHzZVw0KhH5PnmeCJhk8Nxf8B15NktVA_dPKQEsxh858KjhFqbtMRiLzlaqYgFKY-mIxRB_g/embed?start=false&loop=false&delayms=60000

I was getting some weird/random test failure about rendering this doc, so the tests won't render it, but QA and PROD will.

@shaun-technovation shaun-technovation self-assigned this Aug 10, 2021
shaun-technovation added a commit that referenced this issue Aug 10, 2021
Just seeing if this will fix the error on QA:

Unable to load application: RuntimeError: Couldn't find Active Storage
configuration in /app/config/storage.yml

I found the solution here:

https://stackoverflow.com/a/66179690

Refs: #2872
shaun-technovation added a commit that referenced this issue Aug 10, 2021
Running into a "stack level too deep" error:
/GEM_ROOT/gems/scout_apm-2.6.10/lib/scout_apm/layer_children_set.rb:30

Updating Scout to see if that fixes it.

Refs: #2872
@shaun-technovation
Copy link
Contributor

Last week when I was trying to deploy this to QA I was getting a SystemStackError, stack level too deep error.

In the partial backtrace (below), it kept referencing New Relic and Scout.

I ended up removing both New Relic and Scout in order to deploy to QA successfully. We'll probably want to add one or both of these back at some point.


/GEM_ROOT/gems/newrelic_rpm-7.1.0/lib/new_relic/agent/tracer.rb:371 in capture_segment_error
/GEM_ROOT/gems/newrelic_rpm-7.1.0/lib/new_relic/agent/instrumentation/redis/instrumentation.rb:45 in with_tracing
/GEM_ROOT/gems/newrelic_rpm-7.1.0/lib/new_relic/agent/instrumentation/redis/instrumentation.rb:19 in call_with_tracing
/GEM_ROOT/gems/newrelic_rpm-7.1.0/lib/new_relic/agent/instrumentation/redis/prepend.rb:12 in call
/GEM_ROOT/gems/scout_apm-4.1.1/lib/scout_apm/instruments/redis.rb:32 in block in call_with_scout_instruments
/GEM_ROOT/gems/scout_apm-4.1.1/lib/scout_apm/tracer.rb:34 in instrument
/GEM_ROOT/gems/scout_apm-4.1.1/lib/scout_apm/tracer.rb:44 in instrument
/GEM_ROOT/gems/scout_apm-4.1.1/lib/scout_apm/instruments/redis.rb:31 in call_with_scout_instruments
/GEM_ROOT/gems/newrelic_rpm-7.1.0/lib/new_relic/agent/instrumentation/redis/prepend.rb:12 in block in call
/GEM_ROOT/gems/newrelic_rpm-7.1.0/lib/new_relic/agent/instrumentation/redis/instrumentation.rb:19 in block in call_with_tracing
/GEM_ROOT/gems/newrelic_rpm-7.1.0/lib/new_relic/agent/instrumentation/redis/instrumentation.rb:45 in block in with_tracing
/GEM_ROOT/gems/newrelic_rpm-7.1.0/lib/new_relic/agent/tracer.rb:371 in capture_segment_error
/GEM_ROOT/gems/newrelic_rpm-7.1.0/lib/new_relic/agent/instrumentation/redis/instrumentation.rb:45 in with_tracing
/GEM_ROOT/gems/newrelic_rpm-7.1.0/lib/new_relic/agent/instrumentation/redis/instrumentation.rb:19 in call_with_tracing
/GEM_ROOT/gems/newrelic_rpm-7.1.0/lib/new_relic/agent/instrumentation/redis/prepend.rb:12 in call
/GEM_ROOT/gems/scout_apm-4.1.1/lib/scout_apm/instruments/redis.rb:32 in block in call_with_scout_instruments
/GEM_ROOT/gems/scout_apm-4.1.1/lib/scout_apm/tracer.rb:34 in instrument
/GEM_ROOT/gems/scout_apm-4.1.1/lib/scout_apm/tracer.rb:44 in instrument
/GEM_ROOT/gems/scout_apm-4.1.1/lib/scout_apm/instruments/redis.rb:31 in call_with_scout_instruments
/GEM_ROOT/gems/newrelic_rpm-7.1.0/lib/new_relic/agent/instrumentation/redis/prepend.rb:12 in block in call
/GEM_ROOT/gems/newrelic_rpm-7.1.0/lib/new_relic/agent/instrumentation/redis/instrumentation.rb:19 in block in call_with_tracing
/GEM_ROOT/gems/newrelic_rpm-7.1.0/lib/new_relic/agent/instrumentation/redis/instrumentation.rb:45 in block in with_tracing
/GEM_ROOT/gems/newrelic_rpm-7.1.0/lib/new_relic/agent/tracer.rb:371 in capture_segment_error
/GEM_ROOT/gems/newrelic_rpm-7.1.0/lib/new_relic/agent/instrumentation/redis/instrumentation.rb:45 in with_tracing
/GEM_ROOT/gems/newrelic_rpm-7.1.0/lib/new_relic/agent/instrumentation/redis/instrumentation.rb:19 in call_with_tracing
/GEM_ROOT/gems/newrelic_rpm-7.1.0/lib/new_relic/agent/instrumentation/redis/prepend.rb:12 in call
/GEM_ROOT/gems/scout_apm-4.1.1/lib/scout_apm/instruments/redis.rb:32 in block in call_with_scout_instruments
/GEM_ROOT/gems/scout_apm-4.1.1/lib/scout_apm/tracer.rb:34 in instrument
/GEM_ROOT/gems/scout_apm-4.1.1/lib/scout_apm/tracer.rb:44 in instrument
/GEM_ROOT/gems/scout_apm-4.1.1/lib/scout_apm/instruments/redis.rb:31 in call_with_scout_instruments
/GEM_ROOT/gems/newrelic_rpm-7.1.0/lib/new_relic/agent/instrumentation/redis/prepend.rb:12 in block in call
/GEM_ROOT/gems/newrelic_rpm-7.1.0/lib/new_relic/agent/instrumentation/redis/instrumentation.rb:19 in block in call_with_tracing
/GEM_ROOT/gems/newrelic_rpm-7.1.0/lib/new_relic/agent/instrumentation/redis/instrumentation.rb:45 in block in with_tracing
/GEM_ROOT/gems/newrelic_rpm-7.1.0/lib/new_relic/agent/tracer.rb:371 in capture_segment_error
/GEM_ROOT/gems/newrelic_rpm-7.1.0/lib/new_relic/agent/instrumentation/redis/instrumentation.rb:45 in with_tracing
/GEM_ROOT/gems/newrelic_rpm-7.1.0/lib/new_relic/agent/instrumentation/redis/instrumentation.rb:19 in call_with_tracing
/GEM_ROOT/gems/newrelic_rpm-7.1.0/lib/new_relic/agent/instrumentation/redis/prepend.rb:12 in call
/GEM_ROOT/gems/scout_apm-4.1.1/lib/scout_apm/instruments/redis.rb:32 in block in call_with_scout_instruments
/GEM_ROOT/gems/scout_apm-4.1.1/lib/scout_apm/tracer.rb:34 in instrument
/GEM_ROOT/gems/scout_apm-4.1.1/lib/scout_apm/tracer.rb:44 in instrument
/GEM_ROOT/gems/scout_apm-4.1.1/lib/scout_apm/instruments/redis.rb:31 in call_with_scout_instruments
/GEM_ROOT/gems/newrelic_rpm-7.1.0/lib/new_relic/agent/instrumentation/redis/prepend.rb:12 in block in call
/GEM_ROOT/gems/newrelic_rpm-7.1.0/lib/new_relic/agent/instrumentation/redis/instrumentation.rb:19 in block in call_with_tracing
/GEM_ROOT/gems/newrelic_rpm-7.1.0/lib/new_relic/agent/instrumentation/redis/instrumentation.rb:45 in block in with_tracing
/GEM_ROOT/gems/newrelic_rpm-7.1.0/lib/new_relic/agent/tracer.rb:371 in capture_segment_error
/GEM_ROOT/gems/newrelic_rpm-7.1.0/lib/new_relic/agent/instrumentation/redis/instrumentation.rb:45 in with_tracing
/GEM_ROOT/gems/newrelic_rpm-7.1.0/lib/new_relic/agent/instrumentation/redis/instrumentation.rb:19 in call_with_tracing
/GEM_ROOT/gems/newrelic_rpm-7.1.0/lib/new_relic/agent/instrumentation/redis/prepend.rb:12 in call
/GEM_ROOT/gems/scout_apm-4.1.1/lib/scout_apm/instruments/redis.rb:32 in block in call_with_scout_instruments
/GEM_ROOT/gems/scout_apm-4.1.1/lib/scout_apm/tracer.rb:34 in instrument
/GEM_ROOT/gems/scout_apm-4.1.1/lib/scout_apm/tracer.rb:44 in instrument
/GEM_ROOT/gems/scout_apm-4.1.1/lib/scout_apm/instruments/redis.rb:31 in call_with_scout_instruments
/GEM_ROOT/gems/newrelic_rpm-7.1.0/lib/new_relic/agent/instrumentation/redis/prepend.rb:12 in block in call
/GEM_ROOT/gems/newrelic_rpm-7.1.0/lib/new_relic/agent/instrumentation/redis/instrumentation.rb:19 in block in call_with_tracing
/GEM_ROOT/gems/newrelic_rpm-7.1.0/lib/new_relic/agent/instrumentation/redis/instrumentation.rb:45 in block in with_tracing
/GEM_ROOT/gems/newrelic_rpm-7.1.0/lib/new_relic/agent/tracer.rb:371 in capture_segment_error
/GEM_ROOT/gems/newrelic_rpm-7.1.0/lib/new_relic/agent/instrumentation/redis/instrumentation.rb:45 in with_tracing
/GEM_ROOT/gems/newrelic_rpm-7.1.0/lib/new_relic/agent/instrumentation/redis/instrumentation.rb:19 in call_with_tracing
/GEM_ROOT/gems/newrelic_rpm-7.1.0/lib/new_relic/agent/instrumentation/redis/prepend.rb:12 in call
/GEM_ROOT/gems/scout_apm-4.1.1/lib/scout_apm/instruments/redis.rb:32 in block in call_with_scout_instruments
/GEM_ROOT/gems/scout_apm-4.1.1/lib/scout_apm/tracer.rb:34 in instrument
/GEM_ROOT/gems/scout_apm-4.1.1/lib/scout_apm/tracer.rb:44 in instrument
/GEM_ROOT/gems/scout_apm-4.1.1/lib/scout_apm/instruments/redis.rb:31 in call_with_scout_instruments
/GEM_ROOT/gems/newrelic_rpm-7.1.0/lib/new_relic/agent/instrumentation/redis/prepend.rb:12 in block in call
/GEM_ROOT/gems/newrelic_rpm-7.1.0/lib/new_relic/agent/instrumentation/redis/instrumentation.rb:19 in block in call_with_tracing
/GEM_ROOT/gems/newrelic_rpm-7.1.0/lib/new_relic/agent/instrumentation/redis/instrumentation.rb:45 in block in with_tracing
/GEM_ROOT/gems/newrelic_rpm-7.1.0/lib/new_relic/agent/tracer.rb:371 in capture_segment_error
/GEM_ROOT/gems/newrelic_rpm-7.1.0/lib/new_relic/agent/instrumentation/redis/instrumentation.rb:45 in with_tracing
/GEM_ROOT/gems/newrelic_rpm-7.1.0/lib/new_relic/agent/instrumentation/redis/instrumentation.rb:19 in call_with_tracing
/GEM_ROOT/gems/newrelic_rpm-7.1.0/lib/new_relic/agent/instrumentation/redis/prepend.rb:12 in call
/GEM_ROOT/gems/scout_apm-4.1.1/lib/scout_apm/instruments/redis.rb:32 in block in call_with_scout_instruments
/GEM_ROOT/gems/scout_apm-4.1.1/lib/scout_apm/tracer.rb:34 in instrument
/GEM_ROOT/gems/scout_apm-4.1.1/lib/scout_apm/tracer.rb:44 in instrument
/GEM_ROOT/gems/scout_apm-4.1.1/lib/scout_apm/instruments/redis.rb:31 in call_with_scout_instruments
/GEM_ROOT/gems/newrelic_rpm-7.1.0/lib/new_relic/agent/instrumentation/redis/prepend.rb:12 in block in call
/GEM_ROOT/gems/newrelic_rpm-7.1.0/lib/new_relic/agent/instrumentation/redis/instrumentation.rb:19 in block in call_with_tracing
/GEM_ROOT/gems/newrelic_rpm-7.1.0/lib/new_relic/agent/instrumentation/redis/instrumentation.rb:45 in block in with_tracing
/GEM_ROOT/gems/newrelic_rpm-7.1.0/lib/new_relic/agent/tracer.rb:371 in capture_segment_error
/GEM_ROOT/gems/newrelic_rpm-7.1.0/lib/new_relic/agent/instrumentation/redis/instrumentation.rb:45 in with_tracing
/GEM_ROOT/gems/newrelic_rpm-7.1.0/lib/new_relic/agent/instrumentation/redis/instrumentation.rb:19 in call_with_tracing
/GEM_ROOT/gems/newrelic_rpm-7.1.0/lib/new_relic/agent/instrumentation/redis/prepend.rb:12 in call
/GEM_ROOT/gems/scout_apm-4.1.1/lib/scout_apm/instruments/redis.rb:32 in block in call_with_scout_instruments
/GEM_ROOT/gems/scout_apm-4.1.1/lib/scout_apm/tracer.rb:34 in instrument
/GEM_ROOT/gems/scout_apm-4.1.1/lib/scout_apm/tracer.rb:44 in instrument
/GEM_ROOT/gems/scout_apm-4.1.1/lib/scout_apm/instruments/redis.rb:31 in call_with_scout_instruments
/GEM_ROOT/gems/newrelic_rpm-7.1.0/lib/new_relic/agent/instrumentation/redis/prepend.rb:12 in block in call
/GEM_ROOT/gems/newrelic_rpm-7.1.0/lib/new_relic/agent/instrumentation/redis/instrumentation.rb:19 in block in call_with_tracing
/GEM_ROOT/gems/newrelic_rpm-7.1.0/lib/new_relic/agent/instrumentation/redis/instrumentation.rb:45 in block in with_tracing
/GEM_ROOT/gems/newrelic_rpm-7.1.0/lib/new_relic/agent/tracer.rb:371 in capture_segment_error
/GEM_ROOT/gems/newrelic_rpm-7.1.0/lib/new_relic/agent/instrumentation/redis/instrumentation.rb:45 in with_tracing
/GEM_ROOT/gems/newrelic_rpm-7.1.0/lib/new_relic/agent/instrumentation/redis/instrumentation.rb:19 in call_with_tracing
/GEM_ROOT/gems/newrelic_rpm-7.1.0/lib/new_relic/agent/instrumentation/redis/prepend.rb:12 in call
/GEM_ROOT/gems/scout_apm-4.1.1/lib/scout_apm/instruments/redis.rb:32 in block in call_with_scout_instruments
/GEM_ROOT/gems/scout_apm-4.1.1/lib/scout_apm/tracer.rb:34 in instrument
/GEM_ROOT/gems/scout_apm-4.1.1/lib/scout_apm/tracer.rb:44 in instrument
/GEM_ROOT/gems/scout_apm-4.1.1/lib/scout_apm/instruments/redis.rb:31 in call_with_scout_instruments
/GEM_ROOT/gems/newrelic_rpm-7.1.0/lib/new_relic/agent/instrumentation/redis/prepend.rb:12 in block in call
/GEM_ROOT/gems/newrelic_rpm-7.1.0/lib/new_relic/agent/instrumentation/redis/instrumentation.rb:19 in block in call_with_tracing
/GEM_ROOT/gems/newrelic_rpm-7.1.0/lib/new_relic/agent/instrumentation/redis/instrumentation.rb:45 in block in with_tracing
/GEM_ROOT/gems/newrelic_rpm-7.1.0/lib/new_relic/agent/tracer.rb:371 in capture_segment_error
/GEM_ROOT/gems/newrelic_rpm-7.1.0/lib/new_relic/agent/instrumentation/redis/instrumentation.rb:45 in with_tracing
/GEM_ROOT/gems/newrelic_rpm-7.1.0/lib/new_relic/agent/instrumentation/redis/instrumentation.rb:19 in call_with_tracing
/GEM_ROOT/gems/newrelic_rpm-7.1.0/lib/new_relic/agent/instrumentation/redis/prepend.rb:12 in call
/GEM_ROOT/gems/scout_apm-4.1.1/lib/scout_apm/instruments/redis.rb:32 in block in call_with_scout_instruments
/GEM_ROOT/gems/scout_apm-4.1.1/lib/scout_apm/tracer.rb:34 in instrument
/GEM_ROOT/gems/scout_apm-4.1.1/lib/scout_apm/tracer.rb:44 in instrument
/GEM_ROOT/gems/scout_apm-4.1.1/lib/scout_apm/instruments/redis.rb:31 in call_with_scout_instruments
/GEM_ROOT/gems/newrelic_rpm-7.1.0/lib/new_relic/agent/instrumentation/redis/prepend.rb:12 in block in call
/GEM_ROOT/gems/newrelic_rpm-7.1.0/lib/new_relic/agent/instrumentation/redis/instrumentation.rb:19 in block in call_with_tracing
/GEM_ROOT/gems/newrelic_rpm-7.1.0/lib/new_relic/agent/instrumentation/redis/instrumentation.rb:45 in block in with_tracing
/GEM_ROOT/gems/newrelic_rpm-7.1.0/lib/new_relic/agent/tracer.rb:371 in capture_segment_error
/GEM_ROOT/gems/newrelic_rpm-7.1.0/lib/new_relic/agent/instrumentation/redis/instrumentation.rb:45 in with_tracing
/GEM_ROOT/gems/newrelic_rpm-7.1.0/lib/new_relic/agent/instrumentation/redis/instrumentation.rb:19 in call_with_tracing
/GEM_ROOT/gems/newrelic_rpm-7.1.0/lib/new_relic/agent/instrumentation/redis/prepend.rb:12 in call
/GEM_ROOT/gems/scout_apm-4.1.1/lib/scout_apm/instruments/redis.rb:32 in block in call_with_scout_instruments
/GEM_ROOT/gems/scout_apm-4.1.1/lib/scout_apm/tracer.rb:34 in instrument
/GEM_ROOT/gems/scout_apm-4.1.1/lib/scout_apm/tracer.rb:44 in instrument
/GEM_ROOT/gems/scout_apm-4.1.1/lib/scout_apm/instruments/redis.rb:31 in call_with_scout_instruments
/GEM_ROOT/gems/newrelic_rpm-7.1.0/lib/new_relic/agent/instrumentation/redis/prepend.rb:12 in block in call
/GEM_ROOT/gems/newrelic_rpm-7.1.0/lib/new_relic/agent/instrumentation/redis/instrumentation.rb:19 in block in call_with_tracing
/GEM_ROOT/gems/newrelic_rpm-7.1.0/lib/new_relic/agent/instrumentation/redis/instrumentation.rb:45 in block in with_tracing
/GEM_ROOT/gems/newrelic_rpm-7.1.0/lib/new_relic/agent/tracer.rb:371 in capture_segment_error
/GEM_ROOT/gems/newrelic_rpm-7.1.0/lib/new_relic/agent/instrumentation/redis/instrumentation.rb:45 in with_tracing
/GEM_ROOT/gems/newrelic_rpm-7.1.0/lib/new_relic/agent/instrumentation/redis/instrumentation.rb:19 in call_with_tracing
/GEM_ROOT/gems/newrelic_rpm-7.1.0/lib/new_relic/agent/instrumentation/redis/prepend.rb:12 in call
/GEM_ROOT/gems/scout_apm-4.1.1/lib/scout_apm/instruments/redis.rb:32 in block in call_with_scout_instruments
/GEM_ROOT/gems/scout_apm-4.1.1/lib/scout_apm/tracer.rb:34 in instrument
/GEM_ROOT/gems/scout_apm-4.1.1/lib/scout_apm/tracer.rb:44 in instrument
/GEM_ROOT/gems/scout_apm-4.1.1/lib/scout_apm/instruments/redis.rb:31 in call_with_scout_instruments
/GEM_ROOT/gems/newrelic_rpm-7.1.0/lib/new_relic/agent/instrumentation/redis/prepend.rb:12 in block in call
/GEM_ROOT/gems/newrelic_rpm-7.1.0/lib/new_relic/agent/instrumentation/redis/instrumentation.rb:19 in block in call_with_tracing
/GEM_ROOT/gems/newrelic_rpm-7.1.0/lib/new_relic/agent/instrumentation/redis/instrumentation.rb:45 in block in with_tracing
/GEM_ROOT/gems/newrelic_rpm-7.1.0/lib/new_relic/agent/tracer.rb:371 in capture_segment_error
/GEM_ROOT/gems/newrelic_rpm-7.1.0/lib/new_relic/agent/instrumentation/redis/instrumentation.rb:45 in with_tracing
/GEM_ROOT/gems/newrelic_rpm-7.1.0/lib/new_relic/agent/instrumentation/redis/instrumentation.rb:19 in call_with_tracing
/GEM_ROOT/gems/newrelic_rpm-7.1.0/lib/new_relic/agent/instrumentation/redis/prepend.rb:12 in call
/GEM_ROOT/gems/scout_apm-4.1.1/lib/scout_apm/instruments/redis.rb:32 in block in call_with_scout_instruments
/GEM_ROOT/gems/scout_apm-4.1.1/lib/scout_apm/tracer.rb:34 in instrument
/GEM_ROOT/gems/scout_apm-4.1.1/lib/scout_apm/tracer.rb:44 in instrument
/GEM_ROOT/gems/scout_apm-4.1.1/lib/scout_apm/instruments/redis.rb:31 in call_with_scout_instruments
/GEM_ROOT/gems/newrelic_rpm-7.1.0/lib/new_relic/agent/instrumentation/redis/prepend.rb:12 in block in call
/GEM_ROOT/gems/newrelic_rpm-7.1.0/lib/new_relic/agent/instrumentation/redis/instrumentation.rb:19 in block in call_with_tracing
/GEM_ROOT/gems/newrelic_rpm-7.1.0/lib/new_relic/agent/instrumentation/redis/instrumentation.rb:45 in block in with_tracing
/GEM_ROOT/gems/newrelic_rpm-7.1.0/lib/new_relic/agent/tracer.rb:371 in capture_segment_error
/GEM_ROOT/gems/newrelic_rpm-7.1.0/lib/new_relic/agent/instrumentation/redis/instrumentation.rb:45 in with_tracing
/GEM_ROOT/gems/newrelic_rpm-7.1.0/lib/new_relic/agent/instrumentation/redis/instrumentation.rb:19 in call_with_tracing
/GEM_ROOT/gems/newrelic_rpm-7.1.0/lib/new_relic/agent/instrumentation/redis/prepend.rb:12 in call
/GEM_ROOT/gems/scout_apm-4.1.1/lib/scout_apm/instruments/redis.rb:32 in block in call_with_scout_instruments
/GEM_ROOT/gems/scout_apm-4.1.1/lib/scout_apm/tracer.rb:34 in instrument
/GEM_ROOT/gems/scout_apm-4.1.1/lib/scout_apm/tracer.rb:44 in instrument
/GEM_ROOT/gems/scout_apm-4.1.1/lib/scout_apm/instruments/redis.rb:31 in call_with_scout_instruments
/GEM_ROOT/gems/newrelic_rpm-7.1.0/lib/new_relic/agent/instrumentation/redis/prepend.rb:12 in block in call
/GEM_ROOT/gems/newrelic_rpm-7.1.0/lib/new_relic/agent/instrumentation/redis/instrumentation.rb:19 in block in call_with_tracing
/GEM_ROOT/gems/newrelic_rpm-7.1.0/lib/new_relic/agent/instrumentation/redis/instrumentation.rb:45 in block in with_tracing
/GEM_ROOT/gems/newrelic_rpm-7.1.0/lib/new_relic/agent/tracer.rb:371 in capture_segment_error
/GEM_ROOT/gems/newrelic_rpm-7.1.0/lib/new_relic/agent/instrumentation/redis/instrumentation.rb:45 in with_tracing
/GEM_ROOT/gems/newrelic_rpm-7.1.0/lib/new_relic/agent/instrumentation/redis/instrumentation.rb:19 in call_with_tracing
/GEM_ROOT/gems/newrelic_rpm-7.1.0/lib/new_relic/agent/instrumentation/redis/prepend.rb:12 in call
/GEM_ROOT/gems/scout_apm-4.1.1/lib/scout_apm/instruments/redis.rb:32 in block in call_with_scout_instruments
/GEM_ROOT/gems/scout_apm-4.1.1/lib/scout_apm/tracer.rb:34 in instrument
/GEM_ROOT/gems/scout_apm-4.1.1/lib/scout_apm/tracer.rb:44 in instrument
/GEM_ROOT/gems/scout_apm-4.1.1/lib/scout_apm/instruments/redis.rb:31 in call_with_scout_instruments
/GEM_ROOT/gems/newrelic_rpm-7.1.0/lib/new_relic/agent/instrumentation/redis/prepend.rb:12 in block in call
/GEM_ROOT/gems/newrelic_rpm-7.1.0/lib/new_relic/agent/instrumentation/redis/instrumentation.rb:19 in block in call_with_tracing
/GEM_ROOT/gems/newrelic_rpm-7.1.0/lib/new_relic/agent/instrumentation/redis/instrumentation.rb:45 in block in with_tracing
/GEM_ROOT/gems/newrelic_rpm-7.1.0/lib/new_relic/agent/tracer.rb:371 in capture_segment_error
/GEM_ROOT/gems/newrelic_rpm-7.1.0/lib/new_relic/agent/instrumentation/redis/instrumentation.rb:45 in with_tracing
/GEM_ROOT/gems/newrelic_rpm-7.1.0/lib/new_relic/agent/instrumentation/redis/instrumentation.rb:19 in call_with_tracing
/GEM_ROOT/gems/newrelic_rpm-7.1.0/lib/new_relic/agent/instrumentation/redis/prepend.rb:12 in call
/GEM_ROOT/gems/scout_apm-4.1.1/lib/scout_apm/instruments/redis.rb:32 in block in call_with_scout_instruments
/GEM_ROOT/gems/scout_apm-4.1.1/lib/scout_apm/tracer.rb:34 in instrument
/GEM_ROOT/gems/scout_apm-4.1.1/lib/scout_apm/tracer.rb:44 in instrument
/GEM_ROOT/gems/scout_apm-4.1.1/lib/scout_apm/instruments/redis.rb:31 in call_with_scout_instruments
/GEM_ROOT/gems/newrelic_rpm-7.1.0/lib/new_relic/agent/instrumentation/redis/prepend.rb:12 in block in call
/GEM_ROOT/gems/newrelic_rpm-7.1.0/lib/new_relic/agent/instrumentation/redis/instrumentation.rb:19 in block in call_with_tracing
/GEM_ROOT/gems/newrelic_rpm-7.1.0/lib/new_relic/agent/instrumentation/redis/instrumentation.rb:45 in block in with_tracing
/GEM_ROOT/gems/newrelic_rpm-7.1.0/lib/new_relic/agent/tracer.rb:371 in capture_segment_error
/GEM_ROOT/gems/newrelic_rpm-7.1.0/lib/new_relic/agent/instrumentation/redis/instrumentation.rb:45 in with_tracing
/GEM_ROOT/gems/newrelic_rpm-7.1.0/lib/new_relic/agent/instrumentation/redis/instrumentation.rb:19 in call_with_tracing
/GEM_ROOT/gems/newrelic_rpm-7.1.0/lib/new_relic/agent/instrumentation/redis/prepend.rb:12 in call
/GEM_ROOT/gems/scout_apm-4.1.1/lib/scout_apm/instruments/redis.rb:32 in block in call_with_scout_instruments
/GEM_ROOT/gems/scout_apm-4.1.1/lib/scout_apm/tracer.rb:34 in instrument
/GEM_ROOT/gems/scout_apm-4.1.1/lib/scout_apm/tracer.rb:44 in instrument
/GEM_ROOT/gems/scout_apm-4.1.1/lib/scout_apm/instruments/redis.rb:31 in call_with_scout_instruments
/GEM_ROOT/gems/newrelic_rpm-7.1.0/lib/new_relic/agent/instrumentation/redis/prepend.rb:12 in block in call
/GEM_ROOT/gems/newrelic_rpm-7.1.0/lib/new_relic/agent/instrumentation/redis/instrumentation.rb:19 in block in call_with_tracing
/GEM_ROOT/gems/newrelic_rpm-7.1.0/lib/new_relic/agent/instrumentation/redis/instrumentation.rb:45 in block in with_tracing
/GEM_ROOT/gems/newrelic_rpm-7.1.0/lib/new_relic/agent/tracer.rb:371 in capture_segment_error
/GEM_ROOT/gems/newrelic_rpm-7.1.0/lib/new_relic/agent/instrumentation/redis/instrumentation.rb:45 in with_tracing
/GEM_ROOT/gems/newrelic_rpm-7.1.0/lib/new_relic/agent/instrumentation/redis/instrumentation.rb:19 in call_with_tracing
/GEM_ROOT/gems/newrelic_rpm-7.1.0/lib/new_relic/agent/instrumentation/redis/prepend.rb:12 in call
/GEM_ROOT/gems/scout_apm-4.1.1/lib/scout_apm/instruments/redis.rb:32 in block in call_with_scout_instruments
/GEM_ROOT/gems/scout_apm-4.1.1/lib/scout_apm/tracer.rb:34 in instrument
/GEM_ROOT/gems/scout_apm-4.1.1/lib/scout_apm/tracer.rb:44 in instrument
/GEM_ROOT/gems/scout_apm-4.1.1/lib/scout_apm/instruments/redis.rb:31 in call_with_scout_instruments
/GEM_ROOT/gems/newrelic_rpm-7.1.0/lib/new_relic/agent/instrumentation/redis/prepend.rb:12 in block in call
/GEM_ROOT/gems/newrelic_rpm-7.1.0/lib/new_relic/agent/instrumentation/redis/instrumentation.rb:19 in block in call_with_tracing
/GEM_ROOT/gems/newrelic_rpm-7.1.0/lib/new_relic/agent/instrumentation/redis/instrumentation.rb:45 in block in with_tracing
/GEM_ROOT/gems/newrelic_rpm-7.1.0/lib/new_relic/agent/tracer.rb:371 in capture_segment_error
/GEM_ROOT/gems/newrelic_rpm-7.1.0/lib/new_relic/agent/instrumentation/redis/instrumentation.rb:45 in with_tracing
/GEM_ROOT/gems/newrelic_rpm-7.1.0/lib/new_relic/agent/instrumentation/redis/instrumentation.rb:19 in call_with_tracing
/GEM_ROOT/gems/newrelic_rpm-7.1.0/lib/new_relic/agent/instrumentation/redis/prepend.rb:12 in call
/GEM_ROOT/gems/scout_apm-4.1.1/lib/scout_apm/instruments/redis.rb:32 in block in call_with_scout_instruments
/GEM_ROOT/gems/scout_apm-4.1.1/lib/scout_apm/tracer.rb:34 in instrument
/GEM_ROOT/gems/scout_apm-4.1.1/lib/scout_apm/tracer.rb:44 in instrument
/GEM_ROOT/gems/scout_apm-4.1.1/lib/scout_apm/instruments/redis.rb:31 in call_with_scout_instruments
/GEM_ROOT/gems/newrelic_rpm-7.1.0/lib/new_relic/agent/instrumentation/redis/prepend.rb:12 in block in call
/GEM_ROOT/gems/newrelic_rpm-7.1.0/lib/new_relic/agent/instrumentation/redis/instrumentation.rb:19 in block in call_with_tracing
/GEM_ROOT/gems/newrelic_rpm-7.1.0/lib/new_relic/agent/instrumentation/redis/instrumentation.rb:45 in block in with_tracing
/GEM_ROOT/gems/newrelic_rpm-7.1.0/lib/new_relic/agent/tracer.rb:371 in capture_segment_error
/GEM_ROOT/gems/newrelic_rpm-7.1.0/lib/new_relic/agent/instrumentation/redis/instrumentation.rb:45 in with_tracing
/GEM_ROOT/gems/newrelic_rpm-7.1.0/lib/new_relic/agent/instrumentation/redis/instrumentation.rb:19 in call_with_tracing
/GEM_ROOT/gems/newrelic_rpm-7.1.0/lib/new_relic/agent/instrumentation/redis/prepend.rb:12 in call
/GEM_ROOT/gems/scout_apm-4.1.1/lib/scout_apm/instruments/redis.rb:32 in block in call_with_scout_instruments
/GEM_ROOT/gems/scout_apm-4.1.1/lib/scout_apm/tracer.rb:34 in instrument
/GEM_ROOT/gems/scout_apm-4.1.1/lib/scout_apm/tracer.rb:44 in instrument
/GEM_ROOT/gems/scout_apm-4.1.1/lib/scout_apm/instruments/redis.rb:31 in call_with_scout_instruments
/GEM_ROOT/gems/newrelic_rpm-7.1.0/lib/new_relic/agent/instrumentation/redis/prepend.rb:12 in block in call
/GEM_ROOT/gems/newrelic_rpm-7.1.0/lib/new_relic/agent/instrumentation/redis/instrumentation.rb:19 in block in call_with_tracing
/GEM_ROOT/gems/newrelic_rpm-7.1.0/lib/new_relic/agent/instrumentation/redis/instrumentation.rb:45 in block in with_tracing
/GEM_ROOT/gems/newrelic_rpm-7.1.0/lib/new_relic/agent/tracer.rb:371 in capture_segment_error
/GEM_ROOT/gems/newrelic_rpm-7.1.0/lib/new_relic/agent/instrumentation/redis/instrumentation.rb:45 in with_tracing
/GEM_ROOT/gems/newrelic_rpm-7.1.0/lib/new_relic/agent/instrumentation/redis/instrumentation.rb:19 in call_with_tracing
/GEM_ROOT/gems/newrelic_rpm-7.1.0/lib/new_relic/agent/instrumentation/redis/prepend.rb:12 in call
/GEM_ROOT/gems/scout_apm-4.1.1/lib/scout_apm/instruments/redis.rb:32 in block in call_with_scout_instruments
/GEM_ROOT/gems/scout_apm-4.1.1/lib/scout_apm/tracer.rb:34 in instrument
/GEM_ROOT/gems/scout_apm-4.1.1/lib/scout_apm/tracer.rb:44 in instrument
/GEM_ROOT/gems/scout_apm-4.1.1/lib/scout_apm/instruments/redis.rb:31 in call_with_scout_instruments
/GEM_ROOT/gems/newrelic_rpm-7.1.0/lib/new_relic/agent/instrumentation/redis/prepend.rb:12 in block in call
/GEM_ROOT/gems/newrelic_rpm-7.1.0/lib/new_relic/agent/instrumentation/redis/instrumentation.rb:19 in block in call_with_tracing
/GEM_ROOT/gems/newrelic_rpm-7.1.0/lib/new_relic/agent/instrumentation/redis/instrumentation.rb:45 in block in with_tracing
/GEM_ROOT/gems/newrelic_rpm-7.1.0/lib/new_relic/agent/tracer.rb:371 in capture_segment_error
/GEM_ROOT/gems/newrelic_rpm-7.1.0/lib/new_relic/agent/instrumentation/redis/instrumentation.rb:45 in with_tracing
/GEM_ROOT/gems/newrelic_rpm-7.1.0/lib/new_relic/agent/instrumentation/redis/instrumentation.rb:19 in call_with_tracing
/GEM_ROOT/gems/newrelic_rpm-7.1.0/lib/new_relic/agent/instrumentation/redis/prepend.rb:12 in call
/GEM_ROOT/gems/scout_apm-4.1.1/lib/scout_apm/instruments/redis.rb:32 in block in call_with_scout_instruments
/GEM_ROOT/gems/scout_apm-4.1.1/lib/scout_apm/tracer.rb:34 in instrument
/GEM_ROOT/gems/scout_apm-4.1.1/lib/scout_apm/tracer.rb:44 in instrument
/GEM_ROOT/gems/scout_apm-4.1.1/lib/scout_apm/instruments/redis.rb:31 in call_with_scout_instruments
/GEM_ROOT/gems/newrelic_rpm-7.1.0/lib/new_relic/agent/instrumentation/redis/prepend.rb:12 in block in call
/GEM_ROOT/gems/newrelic_rpm-7.1.0/lib/new_relic/agent/instrumentation/redis/instrumentation.rb:19 in block in call_with_tracing
/GEM_ROOT/gems/newrelic_rpm-7.1.0/lib/new_relic/agent/instrumentation/redis/instrumentation.rb:45 in block in with_tracing
/GEM_ROOT/gems/newrelic_rpm-7.1.0/lib/new_relic/agent/tracer.rb:371 in capture_segment_error
/GEM_ROOT/gems/newrelic_rpm-7.1.0/lib/new_relic/agent/instrumentation/redis/instrumentation.rb:45 in with_tracing
/GEM_ROOT/gems/newrelic_rpm-7.1.0/lib/new_relic/agent/instrumentation/redis/instrumentation.rb:19 in call_with_tracing
/GEM_ROOT/gems/newrelic_rpm-7.1.0/lib/new_relic/agent/instrumentation/redis/prepend.rb:12 in call
/GEM_ROOT/gems/scout_apm-4.1.1/lib/scout_apm/instruments/redis.rb:32 in block in call_with_scout_instruments
/GEM_ROOT/gems/scout_apm-4.1.1/lib/scout_apm/tracer.rb:34 in instrument
/GEM_ROOT/gems/scout_apm-4.1.1/lib/scout_apm/tracer.rb:44 in instrument
/GEM_ROOT/gems/scout_apm-4.1.1/lib/scout_apm/instruments/redis.rb:31 in call_with_scout_instruments
/GEM_ROOT/gems/newrelic_rpm-7.1.0/lib/new_relic/agent/instrumentation/redis/prepend.rb:12 in block in call
/GEM_ROOT/gems/newrelic_rpm-7.1.0/lib/new_relic/agent/instrumentation/redis/instrumentation.rb:19 in block in call_with_tracing
/GEM_ROOT/gems/newrelic_rpm-7.1.0/lib/new_relic/agent/instrumentation/redis/instrumentation.rb:45 in block in with_tracing
/GEM_ROOT/gems/newrelic_rpm-7.1.0/lib/new_relic/agent/tracer.rb:371 in capture_segment_error
/GEM_ROOT/gems/newrelic_rpm-7.1.0/lib/new_relic/agent/instrumentation/redis/instrumentation.rb:45 in with_tracing
/GEM_ROOT/gems/newrelic_rpm-7.1.0/lib/new_relic/agent/instrumentation/redis/instrumentation.rb:19 in call_with_tracing
/GEM_ROOT/gems/newrelic_rpm-7.1.0/lib/new_relic/agent/instrumentation/redis/prepend.rb:12 in call
/GEM_ROOT/gems/scout_apm-4.1.1/lib/scout_apm/instruments/redis.rb:32 in block in call_with_scout_instruments
/GEM_ROOT/gems/scout_apm-4.1.1/lib/scout_apm/tracer.rb:34 in instrument
/GEM_ROOT/gems/scout_apm-4.1.1/lib/scout_apm/tracer.rb:44 in instrument
/GEM_ROOT/gems/scout_apm-4.1.1/lib/scout_apm/instruments/redis.rb:31 in call_with_scout_instruments
/GEM_ROOT/gems/newrelic_rpm-7.1.0/lib/new_relic/agent/instrumentation/redis/prepend.rb:12 in block in call
/GEM_ROOT/gems/newrelic_rpm-7.1.0/lib/new_relic/agent/instrumentation/redis/instrumentation.rb:19 in block in call_with_tracing
/GEM_ROOT/gems/newrelic_rpm-7.1.0/lib/new_relic/agent/instrumentation/redis/instrumentation.rb:45 in block in with_tracing
/GEM_ROOT/gems/newrelic_rpm-7.1.0/lib/new_relic/agent/tracer.rb:371 in capture_segment_error
/GEM_ROOT/gems/newrelic_rpm-7.1.0/lib/new_relic/agent/instrumentation/redis/instrumentation.rb:45 in with_tracing
/GEM_ROOT/gems/newrelic_rpm-7.1.0/lib/new_relic/agent/instrumentation/redis/instrumentation.rb:19 in call_with_tracing
/GEM_ROOT/gems/newrelic_rpm-7.1.0/lib/new_relic/agent/instrumentation/redis/prepend.rb:12 in call

@shaun-technovation
Copy link
Contributor

I mentioned last week on our daily stand-up, when we deploy this to production I think we should have a deployment plan, for things like:

  • A rollback plan
  • Things to test post-production
  • A window of time where production will be down
  • An announcement that production will be down?

@rsgonzal
Copy link
Collaborator Author

Like I mentioned in stand, I'm not getting emails ( including parent consent)

@rsgonzal
Copy link
Collaborator Author

rsgonzal commented Sep 1, 2021

I'm not sure new accounts are created properly? I'm looking at dataclips to try and find the ones I created last week and I don't see them. I can't remember those emails..

But Christina just created this account and it is not showing up in the admin but she can login: christina+m1@technovation.org

Update: found mine roxana+mentor826@technovation.org and i believe i made roxana+826@technovation.org as well

shaun-technovation added a commit that referenced this issue Sep 1, 2021
Apparently this uploads heap dumps to s3, according to this commit,
7d819bd:

```
Add sidekiq profiler middleware that puts periodic heap dumps on s3
```

Anywho, it's causing an error w/ the Rails upgrade stuff, and also we
aren't even using it!

/attribute_accessors.rb:202:in `mattr_accessor'
Sep 01 10:41:59 technovation-qa app/worker.1 /app/lib/sidekiq_profiler.rb:23:in `singleton class'

Refs: #2872
shaun-technovation added a commit that referenced this issue Sep 1, 2021
shaun-technovation added a commit that referenced this issue Sep 1, 2021
`default_retries_exhausted` is no longer supported in Sidekiq, I
couldn't find anything in the Sidekiq changelog about it being removed,
but I looked at the latest code and it's no longer there. So, anywho, I
updated our config to remove it from there.

Refs: #2872
@shaun-technovation
Copy link
Contributor

shaun-technovation commented Sep 1, 2021

Like I mentioned in stand, I'm not getting emails ( including parent consent)

Okay, I think we finally have this one resolved. Emails (and a bunch of other things) get run/sent in background jobs. The jobs weren't running because there were a couple of issues with the job runner (Sidekiq) configuration; I removed two antiquated settings (that were added in 2017) and that seems to have fixed it. There were 96 jobs queued up before the fix and they all seem to have run, so I think we're good (at least as far as this issue is concerned). 👍 😄

@rsgonzal
Copy link
Collaborator Author

rsgonzal commented Sep 1, 2021

Yay! Looks good.

I logged into Checkr and don't see myself. I wonder if QA just isn't hooked up?

UPDATE: I found myself, so it is sending info to checkr but i am pending

UPDATE UPDATE: I did some digging and it looks like when people entered their real SSN on QA, they remained pending 🤔

@rsgonzal
Copy link
Collaborator Author

rsgonzal commented Sep 1, 2021

So i signed up and used the bypass code and it worked 🤷🏻‍♀️

@shaun-technovation
Copy link
Contributor

shaun-technovation commented Sep 2, 2021

The jobs weren't running because there were a couple of issues with the job runner (Sidekiq) configuration; I removed two antiquated settings (that were added in 2017) and that seems to have fixed it.

Just following up with this, here are the two commits with more details about the removed configuration settings:

  1. b431668
  2. ec40859

The first one was basically backing up the stacktrace logs and sending them to S3. I'm pretty sure we don't need this functionality, I have never used it. I guess if we need something like this, we can revisit it then, imho.

The second one was marking jobs as "dead" if a job reached its maximum number of retries. I don't know the history behind this functionality, but Sidekiq already has a "dead" queue that can be accessed from the "Background Jobs" link in the admin. The removed code wasn't doing anything w/ the dead jobs, so afaict we're not missing any functionality by removing this code.

@shaun-technovation
Copy link
Contributor

Yay! Looks good.

I logged into Checkr and don't see myself. I wonder if QA just isn't hooked up?

UPDATE: I found myself, so it is sending info to checkr but i am pending

UPDATE UPDATE: I did some digging and it looks like when people entered their real SSN on QA, they remained pending 🤔

I'm not sure if this behavior is expected or not on QA (I'm not really sure how Checkr is supposed to work and if it's supposed to work differently on QA). Anywho, let me know if there's something I need to look at. 😃

@rsgonzal
Copy link
Collaborator Author

rsgonzal commented Sep 2, 2021

I'm not sure if this behavior is expected or not on QA (I'm not really sure how Checkr is supposed to work and if it's supposed to work differently on QA). Anywho, let me know if there's something I need to look at. 😃

I think it's fine? We will probably want to double check once it hits production?

shaun-technovation added a commit that referenced this issue Sep 14, 2021
Apparently this uploads heap dumps to s3, according to this commit,
7d819bd:

```
Add sidekiq profiler middleware that puts periodic heap dumps on s3
```

Anywho, it's causing an error w/ the Rails upgrade stuff, and also we
aren't even using it!

/attribute_accessors.rb:202:in `mattr_accessor'
Sep 01 10:41:59 technovation-qa app/worker.1 /app/lib/sidekiq_profiler.rb:23:in `singleton class'

Refs: #2872
shaun-technovation added a commit that referenced this issue Sep 14, 2021
`default_retries_exhausted` is no longer supported in Sidekiq, I
couldn't find anything in the Sidekiq changelog about it being removed,
but I looked at the latest code and it's no longer there. So, anywho, I
updated our config to remove it from there.

Refs: #2872
@shaun-technovation
Copy link
Contributor

Woohoo, we finally got this one deployed to production. Thank you everyone for your help with this! 🙌 🥳 🏅 🎊 🎈 🎉

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Contractors Work that can be effectively handed off to dev support team ready
Projects
None yet
Development

No branches or pull requests

3 participants