Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

2.x: Upgrade to Gradle 4.3.1, add TakeUntilPerf #6029

Merged
merged 1 commit into from
May 30, 2018

Conversation

akarnokd
Copy link
Member

The JMH plugin 0.4.4 has some shortcomings that prevent the execution of unit tests on Windows 10 (and on some very restricted Linuxes). Version 0.4.5 has been fixed in this regard but it also requires Gradle 4.3.x. Unfortunately, there are no newer versions to the JMH plugin so this is likely as far as we can go with versions.

I've also added a new benchmark to measure the overhead in takeUntil. Here are the results:

i7 4790, Windows 10 x64, Java 8u172, JMH 1.20:

Benchmark                        Mode  Cnt         Score       Error  Units
TakeUntilPerf.flowable          thrpt    5    341933,112   13328,573  ops/s
TakeUntilPerf.flowable:items    thrpt    5  21128765,548  677544,832  ops/s
TakeUntilPerf.observable        thrpt    5    366931,108    8202,644  ops/s
TakeUntilPerf.observable:items  thrpt    5  10733841,643  112853,358  ops/s

The observable here uses an older algorithms & structure and is generally relaying half the items under the same time amonut than the flowable version. PR #6028 can then be evaluated with this benchmark.

@codecov
Copy link

codecov bot commented May 30, 2018

Codecov Report

Merging #6029 into 2.x will decrease coverage by 0.04%.
The diff coverage is n/a.

Impacted file tree graph

@@             Coverage Diff              @@
##                2.x    #6029      +/-   ##
============================================
- Coverage     98.32%   98.27%   -0.05%     
- Complexity     6171     6176       +5     
============================================
  Files           665      665              
  Lines         44726    44726              
  Branches       6206     6206              
============================================
- Hits          43975    43956      -19     
- Misses          219      232      +13     
- Partials        532      538       +6
Impacted Files Coverage Δ Complexity Δ
...l/operators/observable/ObservableFlatMapMaybe.java 88.23% <0%> (-7.85%) 2% <0%> (ø)
...nternal/operators/parallel/ParallelReduceFull.java 91.08% <0%> (-3.97%) 2% <0%> (ø)
...tivex/internal/observers/FutureSingleObserver.java 94.33% <0%> (-3.78%) 24% <0%> (-1%)
...ernal/operators/flowable/FlowableFromIterable.java 93.04% <0%> (-3.21%) 5% <0%> (ø)
...rnal/operators/flowable/FlowableFlatMapSingle.java 92.93% <0%> (-2.72%) 2% <0%> (ø)
...tivex/internal/schedulers/TrampolineScheduler.java 96.1% <0%> (-2.6%) 6% <0%> (ø)
...ava/io/reactivex/processors/BehaviorProcessor.java 96.86% <0%> (-2.25%) 60% <0%> (ø)
...internal/operators/flowable/FlowableSwitchMap.java 95.28% <0%> (-1.89%) 3% <0%> (ø)
...nternal/operators/observable/ObservableCreate.java 97.43% <0%> (-1.71%) 2% <0%> (ø)
...rnal/operators/observable/ObservableSwitchMap.java 95.21% <0%> (-1.07%) 3% <0%> (ø)
... and 22 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 6fefdc6...5a7e7dd. Read the comment docs.

@akarnokd akarnokd merged commit d506ddc into ReactiveX:2.x May 30, 2018
@akarnokd akarnokd deleted the TakeUntilPerf branch May 30, 2018 09:16
@artem-zinnatullin
Copy link
Contributor

TakeUntilPerf.flowable:items    thrpt    5  21128765,548  677544,832  ops/s
TakeUntilPerf.observable:items  thrpt    5  10733841,643  112853,358  ops/s

Holy moly

}
});

while (cdl.getCount() != 0) { }
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@akarnokd I wonder if this loop affects benchmark in positive or negative way, it consumes single core to max

Should cdl.await() be used instead? it parks the thread

I see this pattern in few other benchmarks but idk if it's intentional

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

// It shouldn't affect the comparison though since loop is present in both Flowable and Observable benchmarks

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The problem await is that on my machines they can't wake up that fast and the throughput usually caps out around 200k ops/s, even if the underlying work finished much faster.

For sorter sequences, typically in the range of 1000-10000 items, a spin-wait will result in a much higher throughput number as the end-of-work is detected more eagerly.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ahhhh damn, I think you've mentioned that problem before

What if JMH would be event/callback based so we could benchmark async code like RxJava in a reactive manner?

ie:

@Benchmark
public void flowable(BenchmarkObserver benchmarkObserver) {
    flowable.subscribe(this, Functions.emptyConsumer(), new Action() {
        @Override
        public void run() throws Exception {
            benchmarkObserver.onComplete();
        }
    });
}

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants