-
Notifications
You must be signed in to change notification settings - Fork 12.7k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Consider moving out-of-line module loading from parser to expansion #64197
Comments
This is on my todo list, as it is a per-requisite for moving parser into a library. |
The PR trying to split the libsyntax crate apart (#65324) has to use some hacks due to cfg-expansion being called from the parser. |
#66565 removes those hacks in favor of a less principled approach (revealing the tight coupling between cfg expansion and parsing). |
I would like the ability to start parsing immediately when we see a |
expand: misc cleanups and simplifications Some work I did while trying to understand expand for the purposes of rust-lang#64197. r? @petrochenkov
…ov,eddyb Expansion-driven outline module parsing After this PR, the parser will not do any conditional compilation or loading of external module files when `mod foo;` is encountered. Instead, the parser only leaves `mod foo;` in place in the AST, with no items filled in. Expansion later kicks in and will load the actual files and do the parsing. This entails that the following is now valid: ```rust #[cfg(FALSE)] mod foo { mod bar { mod baz; // `foo/bar/baz.rs` doesn't exist, but no error! } } ``` Fixes rust-lang#64197. r? @petrochenkov
…ov,eddyb Expansion-driven outline module parsing After this PR, the parser will not do any conditional compilation or loading of external module files when `mod foo;` is encountered. Instead, the parser only leaves `mod foo;` in place in the AST, with no items filled in. Expansion later kicks in and will load the actual files and do the parsing. This entails that the following is now valid: ```rust #[cfg(FALSE)] mod foo { mod bar { mod baz; // `foo/bar/baz.rs` doesn't exist, but no error! } } ``` Fixes rust-lang#64197. r? @petrochenkov
…ov,eddyb Expansion-driven outline module parsing After this PR, the parser will not do any conditional compilation or loading of external module files when `mod foo;` is encountered. Instead, the parser only leaves `mod foo;` in place in the AST, with no items filled in. Expansion later kicks in and will load the actual files and do the parsing. This entails that the following is now valid: ```rust #[cfg(FALSE)] mod foo { mod bar { mod baz; // `foo/bar/baz.rs` doesn't exist, but no error! } } ``` Fixes rust-lang#64197. r? @petrochenkov
Expansion-driven outline module parsing After this PR, the parser will not do any conditional compilation or loading of external module files when `mod foo;` is encountered. Instead, the parser only leaves `mod foo;` in place in the AST, with no items filled in. Expansion later kicks in and will load the actual files and do the parsing. This entails that the following is now valid: ```rust #[cfg(FALSE)] mod foo { mod bar { mod baz; // `foo/bar/baz.rs` doesn't exist, but no error! } } ``` Fixes rust-lang#64197. r? @petrochenkov
Expansion-driven outline module parsing After this PR, the parser will not do any conditional compilation or loading of external module files when `mod foo;` is encountered. Instead, the parser only leaves `mod foo;` in place in the AST, with no items filled in. Expansion later kicks in and will load the actual files and do the parsing. This entails that the following is now valid: ```rust #[cfg(FALSE)] mod foo { mod bar { mod baz; // `foo/bar/baz.rs` doesn't exist, but no error! } } ``` Fixes rust-lang#64197. r? @petrochenkov
Expansion-driven outline module parsing After this PR, the parser will not do any conditional compilation or loading of external module files when `mod foo;` is encountered. Instead, the parser only leaves `mod foo;` in place in the AST, with no items filled in. Expansion later kicks in and will load the actual files and do the parsing. This entails that the following is now valid: ```rust #[cfg(FALSE)] mod foo { mod bar { mod baz; // `foo/bar/baz.rs` doesn't exist, but no error! } } ``` Fixes rust-lang#64197. r? @petrochenkov
…e, r=petrochenkov Cleanup stale 'FIXME(rust-lang#64197)' (My first PR in rust-lang, any feedback is welcome. Please don't hesitate to let me know if I'm trying to do something pointless!) Trivial cleanup of a stale `FIXME`, `StringReader.pos` is no longer exposed. For testing added `pos()` method that returns cloned value of `pos`.
Rollup of 9 pull requests Successful merges: - rust-lang#71662 (Implement FromStr for OsString) - rust-lang#71677 (Add explicit references to the BuildHasher trait) - rust-lang#71724 (Doc alias improvements) - rust-lang#71948 (Suggest to await future before ? operator) - rust-lang#72090 (rustc_driver: factor out computing the exit code) - rust-lang#72206 (Cleanup stale 'FIXME(rust-lang#64197)') - rust-lang#72218 (make sure even unleashed miri does not do pointer stuff) - rust-lang#72220 ([const-prop] Don't replace Rvalues that are already constants) - rust-lang#72224 (doc: add links to rotate_(left|right)) Failed merges: r? @ghost
There are some test cases involving `parse` and `tokenstream` and `mut_visit` that are located in `rustc_expand`. Because it used to be the case that constructing a `ParseSess` required the involvement of `rustc_expand`. However, since rust-lang#64197 merged (a long time ago) `rust_expand` no longer needs to be involved. This commit moves the tests into `rustc_parse`. This is the optimal place for the `parse` tests. It's not ideal for the `tokenstream` and `mut_visit` tests -- they would be better in `rustc_ast` -- but they still rely on parsing, which is not available in `rustc_ast`. But `rustc_parse` is lower down in the crate graph and closer to `rustc_ast` than `rust_expand`, so it's still an improvement for them. The exact renaming is as follows: - rustc_expand/src/mut_visit/tests.rs -> rustc_parse/src/parser/mut_visit/tests.rs - rustc_expand/src/tokenstream/tests.rs -> rustc_parse/src/parser/tokenstream/tests.rs - rustc_expand/src/tests.rs + rustc_expand/src/parse/tests.rs -> compiler/rustc_parse/src/parser/tests.rs The latter two test files are combined because there's no need for them to be separate, and having a `rustc_parse::parser::parse` module would be weird. This also means some `pub(crate)`s can be removed.
There are some test cases involving `parse` and `tokenstream` and `mut_visit` that are located in `rustc_expand`. Because it used to be the case that constructing a `ParseSess` required the involvement of `rustc_expand`. However, since rust-lang#64197 merged (a long time ago) `rust_expand` no longer needs to be involved. This commit moves the tests into `rustc_parse`. This is the optimal place for the `parse` tests. It's not ideal for the `tokenstream` and `mut_visit` tests -- they would be better in `rustc_ast` -- but they still rely on parsing, which is not available in `rustc_ast`. But `rustc_parse` is lower down in the crate graph and closer to `rustc_ast` than `rust_expand`, so it's still an improvement for them. The exact renaming is as follows: - rustc_expand/src/mut_visit/tests.rs -> rustc_parse/src/parser/mut_visit/tests.rs - rustc_expand/src/tokenstream/tests.rs -> rustc_parse/src/parser/tokenstream/tests.rs - rustc_expand/src/tests.rs + rustc_expand/src/parse/tests.rs -> compiler/rustc_parse/src/parser/tests.rs The latter two test files are combined because there's no need for them to be separate, and having a `rustc_parse::parser::parse` module would be weird. This also means some `pub(crate)`s can be removed.
…ler-errors Move some tests from `rustc_expand` to `rustc_parse`. There are some test cases involving `parse` and `tokenstream` and `mut_visit` that are located in `rustc_expand`. Because it used to be the case that constructing a `ParseSess` required the involvement of `rustc_expand`. However, since rust-lang#64197 merged (a long time ago) `rust_expand` no longer needs to be involved. This commit moves the tests into `rustc_parse`. This is the optimal place for the `parse` tests. It's not ideal for the `tokenstream` and `mut_visit` tests -- they would be better in `rustc_ast` -- but they still rely on parsing, which is not available in `rustc_ast`. But `rustc_parse` is lower down in the crate graph and closer to `rustc_ast` than `rust_expand`, so it's still an improvement for them. The exact renaming is as follows: - rustc_expand/src/mut_visit/tests.rs -> rustc_parse/src/parser/mut_visit/tests.rs - rustc_expand/src/tokenstream/tests.rs -> rustc_parse/src/parser/tokenstream/tests.rs - rustc_expand/src/tests.rs + rustc_expand/src/parse/tests.rs -> compiler/rustc_parse/src/parser/tests.rs The latter two test files are combined because there's no need for them to be separate, and having a `rustc_parse::parser::parse` module would be weird. This also means some `pub(crate)`s can be removed. r? `@compiler-errors`
Due to module loading parser has to do some not very appropriate for a parser things like
cfg
s on modules (duplicated with expansion)We also don't really need to load those files until we need to incorporate their contents into
NodeId
-entified AST, which happens during expansion.The move will also mean never loading files in unconfigured code.
The loading is currently skipped only if
cfg
is placed directly on themod m;
, but not in other cases likeThe text was updated successfully, but these errors were encountered: