add_child seems very slow under pure java implementation #692
Comments
|
Hmmm. Yes, pure Java version is really really slow. There might be something to improve performance. So, I'll have a look. |
|
This might help: launching the same script with the Java profiler turned on (-J-Xprof flag) returns the following |
|
It seems while add_child is slow, add_next_sibling to the last child of a parent is not. Here is some benchmarking code showing builder usage, direct Doc manipulation, using add_next_sibling and an approach that adds a first child followed by using add_next_sibling. I'm not yet sure why add_next_sibling to the last element of a parent is faster than calling add_child (or if there is any side effect I'm not aware of) require 'nokogiri'
require 'digest'
require 'benchmark'
XML_ELEMENT_COUNT=500
BENCHMARK_LOOP_COUNT=500
# Plain old builder...
def builder
xml = Nokogiri::XML::Builder.new
xml.main do
XML_ELEMENT_COUNT.times do
xml.foobar({:test => 42, :thing => "other thing"})
end
end
xml
end
# Adds the first element with add_child, then uses add_next_sibling
def direct_sibling
doc = Nokogiri::XML::Document.new
root_node = doc << doc.create_element("main")
first_node = root_node.add_child doc.create_element("foobar", {:test => 42, :thing => "other thing"})
(XML_ELEMENT_COUNT-1).times do
first_node.add_next_sibling doc.create_element("foobar", {:test => 42, :thing => "other thing"})
end
doc
end
# Helper method to more intelligently add a child node
def add_smart(parent, new_node)
last_element = parent.last_element_child
if last_element
last_element.add_next_sibling(new_node)
else
parent << new_node
end
end
# Adds child nodes either as next sibling to the last element or as a child
def direct_sibling_smart
doc = Nokogiri::XML::Document.new
root_node = doc << doc.create_element("main")
XML_ELEMENT_COUNT.times do
new_node = doc.create_element("foobar", {:test => 42, :thing => "other thing"})
add_smart(root_node, new_node)
end
doc
end
# 100% direct XML::Document usage
def direct
doc = Nokogiri::XML::Document.new
root_node = doc << doc.create_element("main")
XML_ELEMENT_COUNT.times do
root_node << doc.create_element("foobar", {:test => 42, :thing => "other thing"})
end
doc
end
Benchmark.bm(1) do |bm|
bm.report("smart:") { BENCHMARK_LOOP_COUNT.times { direct_sibling_smart } }
bm.report("sibling:") { BENCHMARK_LOOP_COUNT.times { direct_sibling } }
bm.report("direct:") { BENCHMARK_LOOP_COUNT.times { direct } }
bm.report("builder:") { BENCHMARK_LOOP_COUNT.times { builder } }
endHere are some benchmarks: |
|
Ah, I figured out. This is a negative effect of a hach to make pure Java Nokogiri work like libxml version. Apache Xerces doesn't allow to add a bare text, so we wrap the text by a temporary node. After adding a given text with the wrapper node, pure Java Nokogiri takes the wrapper node out from DOM tree. add_child does this hack while add_next_sibling doesn't. I guess this is the reason of the difference in performance. I'll hava a look whether I can improve this logic. |
|
I also noticed that simply removing the call to relink_namespace makes a drastic difference in performance but introduces test failures. I'm not sure the nokogiri_text_wrapper hack is being invoked in either of our examples. |
|
I'm not familiar with this issue, but i have the benchmark on my dev machine and will investigate it. |
|
Verified that the call to
|
|
I finally got time to figure out what's going on. It seems like the JRuby implementation had a bug that caused it to run in O(n2). The |
Hi,
I'm running the following script
Running under 1.9.3-p125 returns the following
... while running on JRuby (1.6.7, 1.7.0.preview1 and head) return uniformly
When running JRuby's profiler, it turns out that 95% of these twenty-something seconds is spent in the add_child method.
This might be an unfortunate but natural side-effect of using the Xerces library, but I haven't found any explicit documentation about this.
The text was updated successfully, but these errors were encountered: