-
Notifications
You must be signed in to change notification settings - Fork 13.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Foundations of location-sensitive polonius #134268
Conversation
This should all be gated away and deactivatd by default, but let's check: @bors try @rust-timer queue |
This comment was marked as resolved.
This comment was marked as resolved.
This comment has been minimized.
This comment has been minimized.
Foundations of location-sensitive polonius I'd to land the prototype I'm describing in the [polonius project goal](rust-lang/rust-project-goals#118). It still is incomplete and naive and terrible but it's working "well enough" to consider landing. I'd also like to make review easier by not opening a huge PR, but have a couple small-ish ones (the +/- line change summary of this PR looks big, but >80% is moving datalog to a single place). This PR starts laying the foundation for that work: - it refactors and collects 99% of the old datalog fact gen, which was spread around everywhere, into a single dedicated module. It's still present at 3 small places (one of which we should revert anyways) that are kinda deep within localized components that are not as easily extractable into the rest of fact gen, so it's fine for now. - starts introducing the localized constraints, the building blocks of the naive way of implementing the location-sensitive analysis in-tree, which is roughly sketched out in https://smallcultfollowing.com/babysteps/blog/2023/09/22/polonius-part-1/ and https://smallcultfollowing.com/babysteps/blog/2023/09/29/polonius-part-2/ but with a different vibe than per-point environments, just `r1@p: r2@q` constraints. - sets up the skeleton of generating these localized constraints: converting NLL typeck constraints, and creating liveness constraints - introduces the polonius dual to NLL MIR to help development and debugging. It doesn't do much original currently but is a way to see these localized constraints: its an NLL MIR dump + a dumb listing of the constraints. It's not intended to be a long-term thing, it's for testing purposes, and I will replace its contents in the future with a different approach (an HTML where we can more easily explore these constraints, have mermaid graphs of the usual graphviz dumps, etc) I've started documenting the approach in this PR, I'll add more in the future. It's quite simple, and should be very clear when more constraints are introduced anyways. r? `@matthewjasper` Best reviewed per commit so that the datalog move is less bothersome to read, but if you'd prefer we separate that into a different PR, I can do that.
☀️ Try build successful - checks-actions |
This comment has been minimized.
This comment has been minimized.
Finished benchmarking commit (0b8e535): comparison URL. Overall result: no relevant changes - no action neededBenchmarking this pull request likely means that it is perf-sensitive, so we're automatically marking it as not fit for rolling up. While you can manually mark this PR as fit for rollup, we strongly recommend not doing so since this PR may lead to changes in compiler perf. @bors rollup=never Instruction countThis benchmark run did not return any relevant results for this metric. Max RSS (memory usage)Results (primary 4.6%, secondary 2.7%)This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.
CyclesThis benchmark run did not return any relevant results for this metric. Binary sizeThis benchmark run did not return any relevant results for this metric. Bootstrap: 770.238s -> 768.851s (-0.18%) |
Started the review this, but need to step away. Thoughts: First, do you have a branch with all the changes you've made so far? Could be good to link it if so for context. Second, this is still a "big" PR, definitely worth splitting into a couple, I think. That being said, reviewed the first 2 commits and r+ for me on them (if you want to split them out). I'll keep going through the commits as I get time. |
Yes and no, I have a messy older incomplete one and a bunch of local changes because I'm still working on it, so I'm basically cleaning them up as I go to open PRs.
Ok. I'll split the first 2 commits out now, it should simplify this PR a bit. I can split the rest of the old polonius' fact gen later if you or michael want to look at it. (I also rebased to fix conflicts) |
…kh726 A couple of polonius fact generation cleanups This PR is extracted from rust-lang#134268 for easier review and contains its first two commits. They have already been reviewed by `@jackh726.` r? `@jackh726`
Rollup merge of rust-lang#134315 - lqd:polonius-next-episode-1, r=jackh726 A couple of polonius fact generation cleanups This PR is extracted from rust-lang#134268 for easier review and contains its first two commits. They have already been reviewed by `@jackh726.` r? `@jackh726`
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Have reviewed first 8 commits (through "improve consistency..."). A small nit, but otherwise r=me if you want to split them out.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Cursory review of commits 9 onward.
Overall they look fine, but didn't think too deeply so not a full approval.
…kh726 An octuple of polonius fact generation cleanups This PR is extracted from rust-lang#134268 for easier review and contains its first 8 commits. They have already been reviewed by `@jackh726` over there. r? `@jackh726`
I will come back to this for approval of the last few commits once the PR with the first 8 commits lands. |
…kh726 An octuple of polonius fact generation cleanups This PR is extracted from rust-lang#134268 for easier review and contains its first 8 commits. They have already been reviewed by ``@jackh726`` over there. r? ``@jackh726``
…kh726 An octuple of polonius fact generation cleanups This PR is extracted from rust-lang#134268 for easier review and contains its first 8 commits. They have already been reviewed by ```@jackh726``` over there. r? ```@jackh726```
…kh726 An octuple of polonius fact generation cleanups This PR is extracted from rust-lang#134268 for easier review and contains its first 8 commits. They have already been reviewed by ````@jackh726```` over there. r? ````@jackh726````
these are the basic blocks of the naive polonius constraint graph implementation.
this describes the rough algorithm using the localized constraint graph
this will allow calling from polonius MIR
This is mostly for test purposes to show the localized constraints until the MIR debugger is set up.
- move constraints to an Option - check `-Zpolonius=next` only once - rewrite fixme comments to make the actionable part clear
I rebased after #134378 landed. |
r? jackh726 @bors r+ |
☀️ Test successful - checks-actions |
Finished benchmarking commit (426d173): comparison URL. Overall result: ❌ regressions - no action needed@rustbot label: -perf-regression Instruction countThis is the most reliable metric that we have; it was used to determine the overall result at the top of this comment. However, even this metric can sometimes exhibit noise.
Max RSS (memory usage)Results (primary 2.2%)This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.
CyclesResults (primary -4.4%)This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.
Binary sizeThis benchmark run did not return any relevant results for this metric. Bootstrap: 760.428s -> 762.028s (0.21%) |
Encode constraints that hold at all points as logical edges in location-sensitive polonius Currently, with the full setup in rust-lang#134980 (but is from rust-lang#134268), the polonius location-sensitive analysis converts `Locations::All` typeck constraints as edges at all points in the CFG. This was temporary. There's a FIXME about that already, and this PR implements it: we now use the constraints that hold at all points during traversal instead of eagerly materializing them as physical edges. Another easy one `@jackh726.` This fixes the slowness that was happening on the big CFG from the `saturating-float-casts` test (because of its 12M materialized edges) without, AFAICT, simply moving this overhead to traversal: materializing the logical edges is done on-demand. r? `@jackh726` (no rush either)
…kh726 Encode constraints that hold at all points as logical edges in location-sensitive polonius Currently, with the full setup in rust-lang#134980 (but is from rust-lang#134268), the polonius location-sensitive analysis converts `Locations::All` typeck constraints as edges at all points in the CFG. This was temporary. There's a FIXME about that already, and this PR implements it: we now use the constraints that hold at all points during traversal instead of eagerly materializing them as physical edges. Another easy one `@jackh726.` This fixes the slowness that was happening on the big CFG from the `saturating-float-casts` test (because of its 12M materialized edges) without, AFAICT, simply moving this overhead to traversal: materializing the logical edges is done on-demand. r? `@jackh726` (no rush either)
Rollup merge of rust-lang#135290 - lqd:polonius-next-episode-8, r=jackh726 Encode constraints that hold at all points as logical edges in location-sensitive polonius Currently, with the full setup in rust-lang#134980 (but is from rust-lang#134268), the polonius location-sensitive analysis converts `Locations::All` typeck constraints as edges at all points in the CFG. This was temporary. There's a FIXME about that already, and this PR implements it: we now use the constraints that hold at all points during traversal instead of eagerly materializing them as physical edges. Another easy one `@jackh726.` This fixes the slowness that was happening on the big CFG from the `saturating-float-casts` test (because of its 12M materialized edges) without, AFAICT, simply moving this overhead to traversal: materializing the logical edges is done on-demand. r? `@jackh726` (no rush either)
I'd like to land the prototype I'm describing in the polonius project goal. It still is incomplete and naive and terrible but it's working "well enough" to consider landing.
I'd also like to make review easier by not opening a huge PR, but have a couple small-ish ones (the +/- line change summary of this PR looks big, but >80% is moving datalog to a single place).
This PR starts laying the foundation for that work:
r1@p: r2@q
constraints.-Zdump-mir=polonius -Zpolonius=next
. Its current state is not intended to be a long-term thing, just for testing purposes -- I will replace its contents in the future with a different approach (an HTML+js file where we can more easily explore/filter/trace these constraints and loan reachability, have mermaid graphs of the usual graphviz dumps, etc).I've started documenting the approach in this PR, I'll add more in the future. It's quite simple, and should be very clear when more constraints are introduced anyways.
r? @matthewjasper
Best reviewed per commit so that the datalog move is less bothersome to read, but if you'd prefer we separate that into a different PR, I can do that (and michael has offered to review these more mechanical changes if it'd help).