Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add workaround for Ruby garbage collection bug #227

Closed
wants to merge 1 commit into from

Conversation

stanhu
Copy link
Contributor

@stanhu stanhu commented Sep 1, 2021

The Ruby interpreter has a bug in String#concat where the appended array may be garbage collected prematurely because the compiler optimized out a Ruby stack variable. We now call to_ary on the Protobuf object to ensure the array lands on the Ruby stack so the garbage collector sees it.

The real fix in the interpreter is described in https://bugs.ruby-lang.org/issues/18140#note-2, but most current Ruby
interpreters won't have this fix for some time.

The Ruby interpreter has a bug in `String#concat` where the appended
array may be garbage collected prematurely because the compiler
optimized out a Ruby stack variable. We now call `to_ary` on the
Protobuf object to ensure the array lands on the Ruby stack so the
garbage collector sees it.

The real fix in the interpreter is described in
https://bugs.ruby-lang.org/issues/18140#note-2, but most current Ruby
interpreters won't have this fix for some time.
@stanhu
Copy link
Contributor Author

stanhu commented Sep 1, 2021

I considered adding this test:

diff --git a/spec/lib/parse_spec.rb b/spec/lib/parse_spec.rb
index 3705e28..7b4470b 100644
--- a/spec/lib/parse_spec.rb
+++ b/spec/lib/parse_spec.rb
@@ -1609,4 +1609,11 @@ $BODY$
       expect(query.select_tables).to eq(['bar', 'baz'])
     end
   end
+
+  it "parses a query with garbage collection" do
+    GC.stress = true
+    tables = described_class.parse(%(SELECT 1 FROM test WHERE 1 AND 2 AND 3)).tables
+    expect(tables).to eq(['test'])
+    GC.stress = false
+  end
 end

It slows the test down by a few seconds, and it doesn't always fail, depending on the environment and Ruby interpreter you have.

@stanhu
Copy link
Contributor Author

stanhu commented Sep 1, 2021

@lfittl Do you think we should include the test above?

@lfittl
Copy link
Member

lfittl commented Oct 8, 2021

@stanhu Thanks for the contribution & for investing the time to get to the root cause of this! (and for your patience - I was out on leave and missed this PR)

I think we can merge this without a test, since as you note the test slows things down, and isn't fully reliable. I'll add a comment at the top of the method linking the Ruby bug, so its clear why we're doing the .to_ary calls.

@lfittl
Copy link
Member

lfittl commented Oct 8, 2021

Merged in 945cb8f

@lfittl lfittl closed this Oct 8, 2021
@stanhu
Copy link
Contributor Author

stanhu commented Oct 11, 2021

@lfittl Thanks! Would you mind releasing a new version? It seems our team worked around the problem once by disabling garbage collection before running PgQuery#parse, but they ran into another case where they forgot to do this.

@lfittl
Copy link
Member

lfittl commented Oct 15, 2021

@lfittl Thanks! Would you mind releasing a new version? It seems our team worked around the problem once by disabling garbage collection before running PgQuery#parse, but they ran into another case where they forgot to do this.

Yep - Released!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants