Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

make RemainingCandidates::next peekable. #5044

Merged
merged 4 commits into from
Feb 25, 2018

Conversation

Eh2406
Copy link
Contributor

@Eh2406 Eh2406 commented Feb 15, 2018

candidates.next always came with a candidates.clone().next(prev_active).is_ok so let's just make that part of next. no clone needed.

@rust-highfive
Copy link

r? @matklad

(rust_highfive has picked a reviewer for you, use r? to override)

@matklad
Copy link
Member

matklad commented Feb 15, 2018

r? @alexcrichton

`candidates.next` always came with a `candidates.clone().next(prev_active).is_ok` so lets just make that part of `next`. no `clone` needed.
@Eh2406 Eh2406 force-pushed the PeekAtRemainingCandidates branch from 83b167e to 4afbd85 Compare February 21, 2018 20:39
@Eh2406
Copy link
Contributor Author

Eh2406 commented Feb 22, 2018

Thinking about it some more, an alternative is to just skip the has_another code and always push a cloned BacktrackFrame to the backtrack_stack then use the loop/if let in find_candidate to find one with another candidate. Currently that if let is always true, as we only add an item to the backtrack_stack if it has another candidate.

In the happy path this is a small performance hit, a clone for every time we only have 1 candidate. But how often does that happen?

@Eh2406
Copy link
Contributor Author

Eh2406 commented Feb 22, 2018

I started work on that alternative at:
https://github.com/Eh2406/cargo/tree/remove_has_another
Just to compare which leads to more clean up.

how often does that happen?

Actually every time we are forced by a previous selection.

Copy link
Member

@alexcrichton alexcrichton left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok! I'm back from traveling now so had a chance to sit down and read this. Looks great to me, thanks @Eh2406! Just a few tiny nits.

}
}
return Ok(b);
if let Some(r) = ::std::mem::replace(&mut self.has_another, Some(b)) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: could std::mem be imported above?

frame.remaining_candidates.clone().next(prev_active).is_ok(),
)
};
let next= frame.remaining_candidates.next(frame.context_backup.prev_active(&frame.dep));
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: spacing on the = here

@bors
Copy link
Contributor

bors commented Feb 24, 2018

☔ The latest upstream changes (presumably #4978) made this pull request unmergeable. Please resolve the merge conflicts.

@Eh2406
Copy link
Contributor Author

Eh2406 commented Feb 25, 2018

@alexcrichton, I fixed you nits. Then I noted that some of my cleanups from the remove_has_another bach can still be made. Namly, with a small change we can have only 1 place we push to backtrack_stack. IMO this cleans up control flow. I am working on fixing the merge conflicts.

…idates

# Conflicts:
#	src/cargo/core/resolver/mod.rs
@Eh2406
Copy link
Contributor Author

Eh2406 commented Feb 25, 2018

merge done.

@alexcrichton
Copy link
Member

@bors: r+

@bors
Copy link
Contributor

bors commented Feb 25, 2018

📌 Commit 0be926d has been approved by alexcrichton

@bors
Copy link
Contributor

bors commented Feb 25, 2018

⌛ Testing commit 0be926d with merge 6cfd4e2...

bors added a commit that referenced this pull request Feb 25, 2018
make RemainingCandidates::next peekable.

`candidates.next` always came with a `candidates.clone().next(prev_active).is_ok` so let's just make that part of `next`. no `clone` needed.
@bors
Copy link
Contributor

bors commented Feb 25, 2018

☀️ Test successful - status-appveyor, status-travis
Approved by: alexcrichton
Pushing 6cfd4e2 to master...

@bors bors merged commit 0be926d into rust-lang:master Feb 25, 2018
@bors bors mentioned this pull request Feb 25, 2018
@Eh2406 Eh2406 deleted the PeekAtRemainingCandidates branch February 25, 2018 14:24
@alexcrichton
Copy link
Member

Looking at some profiles from master Cargo, I think this commit unfortunately makes us lose out on a previously important optimization which is when has_another was false don't actually clone the Context. That's the fast path when using lock files (and the second round of resolution currently in Cargo), but @Eh2406 do you think we might be able to recover that?

@Eh2406
Copy link
Contributor Author

Eh2406 commented Mar 6, 2018

That is not good. This PR was not intended to change that optimization. I will help track down that regression any way I can. I don't see how this commit could be causing a problem with that, the only Context::clone is line 754 witch is guarded by line 751. And anyway, things in the lock file are activateed way back at line 647.

So I can dig in (when I next have time), what is the exact command you are testing with?
What behavior did you see before? How did you observe it?
What behavior are you seeing now?

With that I can bisect to Identify the problem commit. I suspect it is from #5000 or #5104 as they affected clones of context. In any case, there should be ways to make it all work and have that optimization. Just need to figure out how.

@alexcrichton
Copy link
Member

Er sorry, I think I misdiagnosed! I do indeed believe this is #5000 as it moved the creation of backtrack outside of the conditional (unconditionally cloning before activate).

As for a test case I was running perf record ./x.py build which for bad reasons runs cargo build a bunch of times (~10) which are all noops but work over a pretty big workspace. I found that resolution was taking a pretty big chunk of the Cargo time and I was just poking around reading code and realized that the optimization wasn't there by accident any more!

On master though it's this line --

context_backup: Context::clone(&cx),

which happens unconditionally before activate rather than inside the has_another conditional. Unfortunately this didn't look easy to rejigger :(

@Eh2406
Copy link
Contributor Author

Eh2406 commented Mar 6, 2018

Ya, that makes sense, and will be trickier. I will look into it as soon as work allows.

@alexcrichton
Copy link
Member

Ok awesome, thanks!

@alexcrichton
Copy link
Member

I've opened a tracking issue for this at #5130

@ehuss ehuss added this to the 1.26.0 milestone Feb 6, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants