Make AST->HIR lowering incremental

[cjgillot]

Proposal

AST->HIR lowering is currently done ahead of the construction of the incremental compilation engine. This issue proposes to make lowering incremental by introducing a query lower_to_hir(def: LocalDefId) -> Option<&hir::OwnerInfo<'_>> which lowers the AST item corresponding to definition def into the corresponding HIR and some associated information.

The objective is to eventually skip lowering some parts of the AST, and to allow progress towards making resolution and expansion incremental.

The proposed design is as follows:

• remove global analyses from the current lowering pass, and pull them back to resolution stage;
• attach lowering information to definitions instead of having multiple maps inside hir::Crate, effectively making hir::Crate an IndexVec<LocalDefId, Option<OwnerInfo<'_>>>;
• extract the required resolution outputs from the resolver and keep them immutable;
• allow to create new definitions from inside the query system (*);
• index the AST by LocalDefId, using an IndexVec<LocalDefId, AstOwner> where AstOwner is an enum holding OwningRef<Lrc<Crate>, ItemLike> for different ItemLike;
• perform HIR indexing per-owner;
• make lowering incremental.

One of the most subtle changes is (*), which allows to create new definitions from inside queries. It is made safe using an eval-always query which forces re-execution of its caller, and effectively enforces the side-effects creating the definition. The first time definitions are iterated over (for instance for metadata output), a read-lock to the definitions is leaked, which seals the list of definitions.

Draft branch is accessible at rust-lang/rust@master...cjgillot:lower-incr

Mentors or Reviewers

Process

The main points of the Major Change Process are as follows:

• File an issue describing the proposal.
• A compiler team member or contributor who is knowledgeable in the area can second by writing @rustbot second.
  • Finding a "second" suffices for internal changes. If however, you are proposing a new public-facing feature, such as a -C flag, then full team check-off is required.
  • Compiler team members can initiate a check-off via @rfcbot fcp merge on either the MCP or the PR.
• Once an MCP is seconded, the Final Comment Period begins. If no objections are raised after 10 days, the MCP is considered approved.

You can read more about Major Change Proposals on forge.

Comments

This issue is not meant to be used for technical discussion. There is a Zulip stream for that. Use this issue to leave procedural comments, such as volunteering to review, indicating that you second the proposal (or third, etc), or raising a concern that you would like to be addressed.

[rustbot]

This issue is not meant to be used for technical discussion. There is a Zulip stream for that. Use this issue to leave procedural comments, such as volunteering to review, indicating that you second the proposal (or third, etc), or raising a concern that you would like to be addressed.

cc @rust-lang/compiler @rust-lang/compiler-contributors

[pnkfelix]

@rustbot second

[apiraino]

@rustbot label -final-comment-period +major-change-accepted