Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

resque: be more specific when handling ActiveJob wrapped jobs #710

Closed
wants to merge 1 commit into from

Conversation

isnotajoke
Copy link
Contributor

Goal

The Resque Bugsnag integration adds helpful context and payload information to reports from Resque tasks. A check within the integration attempts to unwrap ActiveJob jobs run via Resque. The check used to perform this unwrapping assumes that all jobs will run via ActiveJob, and causes errors when used with jobs other than ActiveJob whose first argument does not respond sensibly to []. For example:

Error in internal notify block: no implicit conversion of String into Integer

after bubbling up to the top-level Bugsnag notify method. This occurs before any Resque-specific context can be added to the report, and yields reports that are less useful than they could be. We want to prevent these errors so Resque reports have more uniform context.

Design

The comment in the current implementation asserts an expectation about the payload of ActiveJob reports, but doesn't actually assert it or guard the following code by checking that assumption. It seemed reasonable to me to guard the error-producing conditional with an explicit assertion following the comment – restricting it to only those payloads for which it makes sense.

Changeset

  • Guard ActiveJob-specific Resque handling with an assertion that we're handling an ActiveJob report.

Testing

  • Confirmed in my environment that Resque jobs more reliably have Resque context associated with them
  • Confirmed in my environment that "Error in internal notify block" errors (associated with this issue) are reduced/eliminated after this fix
  • Confirmed in my environment that errors originating in ActiveJob tasks are properly unwrapped
  • Added specs for this case

Copy link
Contributor Author

@isnotajoke isnotajoke left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Self-review. I still need to add specs and finish testing this; wanted to open the PR to get feedback on the implementation while I do that.

@@ -49,7 +49,7 @@ def save

# when using Active Job the payload "class" will always be the Resque
# "JobWrapper", not the actual job class so we need to fix this here
if metadata['args'] && metadata['args'][0] && metadata['args'][0]['job_class']
if class_name == 'ActiveJob::QueueAdapters::ResqueAdapter::JobWrapper' && metadata['args'] && metadata['args'][0] && metadata['args'][0]['job_class']
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's a risk here that this may be too specific or accidentally a breaking change – either because it doesn't account for an ActiveJob-specific quirk, or because the existing code was coincidentally unwrapping exceptions from other frameworks. An alternative implementation would be wrapping this existing logic in a begin/rescue; it would still fail, but in a way that is less disruptive than the existing implementation. I'm open to going that way if it's preferred by maintainers

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is a good way to do it 🙂

I was considering checking if ['args'][0] is a hash or responds to [] but, given this is specifically to unwrap the Active Job class, it makes sense to me to make it specific to Active Job

Copy link
Contributor

@imjoehaines imjoehaines left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the PR, @isnotajoke

Sorry for this, we do have tests that run Resque jobs for real, but unfortunately the jobs aren't passed arguments so didn't trigger this bug 😕

@rails_integrations
Scenario: Resque (no on_exit hooks)
When I run "bundle exec rake resque:work" in the rails app
And I run "Resque.enqueue(ResqueWorker)" with the rails runner
And I wait to receive a request
Then the request is valid for the error reporting API version "4.0" for the "Ruby Bugsnag Notifier"
And the event "unhandled" is true
And the event "context" equals "ResqueWorker@crash"
And the event "severity" equals "error"
And the event "severityReason.type" equals "unhandledExceptionMiddleware"
And the event "severityReason.attributes.framework" equals "Resque"
And the event "app.type" equals "resque"
And the exception "errorClass" equals "RuntimeError"
And the event "metaData.config.delivery_method" equals "synchronous"
And the event "metaData.context" equals "ResqueWorker@crash"
And the event "metaData.payload.class" equals "ResqueWorker"
And the event "metaData.rake_task.name" equals "resque:work"
And the event "metaData.rake_task.description" equals "Start a Resque worker"
And the event "metaData.rake_task.arguments" is null
@rails_integrations
Scenario: Resque (with on_exit hooks)
Given I set environment variable "RUN_AT_EXIT_HOOKS" to "1"
When I run "bundle exec rake resque:work" in the rails app
And I run "Resque.enqueue(ResqueWorker)" with the rails runner
And I wait to receive a request
Then the request is valid for the error reporting API version "4.0" for the "Ruby Bugsnag Notifier"
And the event "unhandled" is true
And the event "context" equals "ResqueWorker@crash"
And the event "severity" equals "error"
And the event "severityReason.type" equals "unhandledExceptionMiddleware"
And the event "severityReason.attributes.framework" equals "Resque"
And the event "app.type" equals "resque"
And the exception "errorClass" equals "RuntimeError"
And the event "metaData.config.delivery_method" equals "thread_queue"
And the event "metaData.context" equals "ResqueWorker@crash"
And the event "metaData.payload.class" equals "ResqueWorker"
And the event "metaData.rake_task.name" equals "resque:work"
And the event "metaData.rake_task.description" equals "Start a Resque worker"
And the event "metaData.rake_task.arguments" is null

I'll make sure to change them to have arguments to check for this case and I can add some specs for this as well

@@ -49,7 +49,7 @@ def save

# when using Active Job the payload "class" will always be the Resque
# "JobWrapper", not the actual job class so we need to fix this here
if metadata['args'] && metadata['args'][0] && metadata['args'][0]['job_class']
if class_name == 'ActiveJob::QueueAdapters::ResqueAdapter::JobWrapper' && metadata['args'] && metadata['args'][0] && metadata['args'][0]['job_class']
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is a good way to do it 🙂

I was considering checking if ['args'][0] is a hash or responds to [] but, given this is specifically to unwrap the Active Job class, it makes sense to me to make it specific to Active Job

@imjoehaines
Copy link
Contributor

I'll merge this in #711 where I've added & updated some tests

Thanks again!

@imjoehaines imjoehaines mentioned this pull request Nov 30, 2021
@xljones
Copy link

xljones commented Dec 7, 2021

@isnotajoke this has been released in https://github.com/bugsnag/bugsnag-ruby/releases/tag/v6.24.1 thanks for the contribution! 🎉

@xljones xljones added bug Confirmed bug released This feature/bug fix has been released labels Dec 7, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Confirmed bug released This feature/bug fix has been released
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants