Skip to content

Commit

Permalink
Prevent memory bloat in ActiveJob children
Browse files Browse the repository at this point in the history
`ActiveJob` attaches a single `LogSubscriber` to be notified of events.
When one job enqueues another, that's treated as a child event and
stored on the same stack as the parent. If a large number of children
are enqueued that leads to R14 errors on Heroku.

The fix here is to unsubscribe all existing "enqueue" listeners and
then add a new subscriber for just that event. The original handler
is copied from the parent class using `define_method` so that it's
picked up by `attach_to`.

Related:

rails/rails#21036
rails/rails#5932

Before and after:

![Heroku metrics chart showing 1GB drop in memory usage after this change deployed](https://user-images.githubusercontent.com/47136/28436362-b51b3d1e-6d64-11e7-8b24-fc67927e7081.png)
  • Loading branch information
croaky committed Aug 8, 2017
1 parent efda495 commit 38b530c
Show file tree
Hide file tree
Showing 5 changed files with 39 additions and 0 deletions.
1 change: 1 addition & 0 deletions lib/suspenders.rb
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
require 'suspenders/version'
require 'suspenders/generators/app_generator'
require "suspenders/generators/enforce_ssl_generator"
require "suspenders/generators/initialize_active_job_generator"
require 'suspenders/generators/static_generator'
require 'suspenders/generators/stylesheet_base_generator'
require 'suspenders/actions'
Expand Down
1 change: 1 addition & 0 deletions lib/suspenders/generators/app_generator.rb
Original file line number Diff line number Diff line change
Expand Up @@ -217,6 +217,7 @@ def remove_routes_comment_lines

def generate_default
run("spring stop")
generate("suspenders:initialize_active_job")
generate("suspenders:enforce_ssl")
generate("suspenders:static")
generate("suspenders:stylesheet_base")
Expand Down
19 changes: 19 additions & 0 deletions lib/suspenders/generators/initialize_active_job_generator.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
require "rails/generators"

module Suspenders
class InitializeActiveJobGenerator < Rails::Generators::Base
source_root(
File.expand_path(
File.join("..", "..", "..", "templates"),
File.dirname(__FILE__),
),
)

def initialize_active_job
copy_file(
"active_job.rb",
"config/initializers/active_job.rb",
)
end
end
end
5 changes: 5 additions & 0 deletions spec/features/new_project_spec.rb
Original file line number Diff line number Diff line change
Expand Up @@ -100,6 +100,11 @@
expect(env).to include "RACK_MINI_PROFILER=0"
end

it "initializes ActiveJob to avoid memory bloat" do
expect(File).
to exist("#{project_path}/config/initializers/active_job.rb")
end

it "creates a rack-mini-profiler initializer" do
expect(File).
to exist("#{project_path}/config/initializers/rack_mini_profiler.rb")
Expand Down
13 changes: 13 additions & 0 deletions templates/active_job.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
require "active_job/logging"

ActiveSupport::Notifications.unsubscribe("enqueue.active_job")

module ActiveJob
module Logging
class EnqueueLogSubscriber < LogSubscriber
define_method :enqueue, instance_method(:enqueue)
end
end
end

ActiveJob::Logging::EnqueueLogSubscriber.attach_to(:active_job)

0 comments on commit 38b530c

Please sign in to comment.