From 33f9f6d4e9306be240ef14ae325ff1a01ff2149b Mon Sep 17 00:00:00 2001 From: Keegan McAllister Date: Fri, 7 Nov 2014 18:11:52 -0800 Subject: [PATCH 01/27] RFC: Macro reform --- text/0000-macro-reform.md | 211 ++++++++++++++++++++++++++++++++++++++ 1 file changed, 211 insertions(+) create mode 100644 text/0000-macro-reform.md diff --git a/text/0000-macro-reform.md b/text/0000-macro-reform.md new file mode 100644 index 00000000000..2f3897af23f --- /dev/null +++ b/text/0000-macro-reform.md @@ -0,0 +1,211 @@ +- Start Date: 2014-11-05 +- RFC PR: (leave this empty) +- Rust Issue: (leave this empty) + +# Summary + +Various enhancements to `macro_rules!` ahead of its standardization in 1.0. + +# Motivation + +`macro_rules!` has [many rough +edges](https://github.com/rust-lang/rfcs/issues/440). A few of the big ones: + +- You can't re-export macros +- Even if you could, names produced by the re-exported macro won't follow the re-export +- You can't use the same macro in-crate and exported, without the "curious inner-module" hack +- There's no namespacing at all +- You can't control which macros are imported from a crate +- You need the feature-gated `#[phase(plugin)]` to import macros + +These issues in particular are things we have a chance of addressing for 1.0. +This RFC contains plans to do so. + +# Detailed design + +## Rename + +Rename `macro_rules!` to `macro!`, analogous to `fn`, `struct`, etc. + +## `#[visible(...)]` + +`#[macro_export]` and `#[macro_escape]` are replaced with a new attribute +applied to a `macro!` invocation: + +* `#[visible(this_crate)]` — this macro "escapes" up the module hierarchy to + the crate root, so it can be used anywhere in the crate after the definition + (according to a depth-first traversal). This is like putting `#[macro_escape]` + on the module and all its ancestors, but applies *only* to the macro with the + `#[visible]` attribute. + +* `#[visible(other_crates)]` — same meaning as `#[macro_export]` today + +These can be combined as `#[visible(this_crate, other_crates)]` (in either +order). + +The default (as today) is that the macro is only visible within the lexical +scope where it is defined. + +## `$crate` + +Add a special metavar `$crate` which expands to `::foo` when the macro was +imported from crate `foo`, and to nothing when it was defined in-crate. +`$crate::bar::baz` will be an absolute path either way. + +This feature eliminates the need for the "curious inner-module" and also +enables macro re-export (see below). It is [implemented and +tested](https://github.com/kmcallister/rust/commits/macro-reexport) but needs a +rebase. + +## Crate scope for macros + +Instead of a single global namespace for macro definitions, we now have one +namespace per crate. We introduce an attribute to `use` macros with optional +renaming: + +```rust +#[use_macros(std::vec, std::panic as fail)] +``` + +(We'd need to extend attribute syntax, or change this to be compatible.) + +This can be applied to a module, a function, or anywhere else attributes are +allowed. + +There's no way to invoke a macro with a qualified name; this obviates the need +for changes to the grammar / parser. + +This isn't an actual `use` item because macro expansion happens before name +resolution, and libsyntax knows nothing about the latter. + +This change applies to `macro!` and to all other syntax extensions, even +decorators that are used as attributes. + +Many macros expand using other "private macros" as an implementation detail. +For example, librustc's `declare_lint!` uses `lint_initializer!`. The client +should not know about this macro, although it still needs to be exported for +cross-crate use. The solution is to allow `use_macros` on `macro!` itself, and +allow `$crate` in that context: + +```rust +/// Not to be imported directly. +#[visible(other_crates)] +macro! lint_initializer ( ... ) + +/// Declare a lint. +#[visible(other_crates)] +#[use_macros($crate::lint_initializer)] +macro! declare_lint ( + ($name:ident, $level:ident, $desc:expr) => ( + static $name: &'static $crate::lint::Lint + = &lint_initializer!($name, $level, $desc); + ) +) +``` + +The macro `lint_initializer!` will be visible only during further expansion of +the result of invoking `declare_lint!`. + +Procedural macros need their own way to manipulate the expansion context, but +that's an unstable internal API, so it's outside the scope of this RFC. +Ideally the implementation of `macro!` will use the same API. + +## Macro re-export + +With `$crate` we can easily re-export macros that were imported from another +crate. + +For example, libcollections defines a `vec!` macro, which would now look like: + +```rust +#[visible(other_crates)] +macro! vec ( + ($($e:expr),*) => ({ + let mut _temp = $crate::vec::Vec::new(); + $(_temp.push($e);)* + _temp + }) +) +``` + +Currently, libstd duplicates this macro in its own `macros.rs`. Now it could +do + +```rust +#![reexport_macros(collections::vec)] +``` + +as long as the module `std::vec` is interface-compatible with +`collections::vec`. + +(Actually the current libstd `vec!` is completely different for efficiency, but +it's just an example.) + +## Auto-load macros + +Since macros are now crate-scoped, we can load macros from every `extern crate` +without a special attribute. (Probably we should exclude `extern crate`s that +aren't at the crate root, because there's no way `$crate` paths will be +correct.) + +`#[phase(plugin)]` becomes simply `#[plugin]` and is still feature-gated. It +only controls whether to search for and run a plugin registrar function. The +plugin itself will decide whether it's to be linked at runtime, by calling a +`Registry` method. + +In the future I would like to support `#[plugin(... any metas ...)]` where +these "arguments" are available in the registrar and can be used to configure +how the plugin works. This RFC does not cover that feature; I just want to +make sure our design is compatible. + + +# Drawbacks + +This is big churn on a major feature, not long before 1.0. + +We can ship improved versions of `macro!` in a back-compat way (in theory; I +would like to smoke test this idea before 1.0). So we could defer all this +reform until after 1.0. The main reason not to is macro import/export. Right +now every macro you import will be expanded using your local copy of `macro!`, +regardless of what the macro author had in mind. + +# Alternatives + +We could try to implement proper hygienic capture of crate names in macros. +This would be wonderful, but I don't think we can get it done for 1.0. + +We would have to actually parse the macro RHS when it's defined, find all the +paths it wants to emit (somehow), and then turn each crate reference within +such a path into a globally unique thing that will still work when expanded in +another crate. Right now libsyntax is oblivious to librustc's name resolution +rules, and those rules can't be applied until macro expansion is done, because +(for example) a macro can expand to a `use` item. + +nrc suggested dropping the `#![macro_escape]` functionality as part of this +reform. Two ways this could work out: + +- *All* macros are visible throughout the crate. This seems bad; I depend on + module scoping to stay (marginally) sane when working with macros. You can + have private helper macros in two different modules without worrying that + the names will clash. + +- Only macros at the crate root are visible throughout the crate. I'm also + against this because I like keeping `lib.rs` as a declarative description + of crates, modules, etc. without containing any actual code. Forcing the + user's hand as to which file a particular piece of code goes in seems + un-Rusty. + +# Unresolved questions + +Do we support globs in `use_macros`? How does prelude injection work? +How do I get all of libcore's macros if my crate is `no_std`? + +Does `use_macros` also define the name in child modules? In a sense this is the +more natural thing to do in libsyntax, but it's inconsistent with normal `use` +items. + +All of the syntax is bikeshedable. For example, should `use_macros` include +the exclamation point? What about when it applies to item decorator attributes? + +Allowing `$crate` in attributes is weird. Maybe we should use some other +syntax that fits with existing attribute parsing. From 4b5e09fffe15435db3c962e4700634bdaa764fdf Mon Sep 17 00:00:00 2001 From: Keegan McAllister Date: Fri, 7 Nov 2014 20:07:41 -0800 Subject: [PATCH 02/27] Add disclaimer --- text/0000-macro-reform.md | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/text/0000-macro-reform.md b/text/0000-macro-reform.md index 2f3897af23f..8569e834805 100644 --- a/text/0000-macro-reform.md +++ b/text/0000-macro-reform.md @@ -6,6 +6,11 @@ Various enhancements to `macro_rules!` ahead of its standardization in 1.0. +**Note**: This is not the final Rust macro system design for all time. Rather, +it addresses the largest usability problems within the limited time frame for +1.0. It's my hope that a lot of these problems can be solved in nicer ways +in the long term (there is some discussion of this below). + # Motivation `macro_rules!` has [many rough From 79c74f99f41235337b8f53dc36eea20ca3d0fd57 Mon Sep 17 00:00:00 2001 From: Keegan McAllister Date: Wed, 12 Nov 2014 03:21:48 -0800 Subject: [PATCH 03/27] New syntax proposal --- text/0000-macro-reform.md | 175 ++++++++++++++++++++++++++------------ 1 file changed, 121 insertions(+), 54 deletions(-) diff --git a/text/0000-macro-reform.md b/text/0000-macro-reform.md index 8569e834805..d9c07c6177f 100644 --- a/text/0000-macro-reform.md +++ b/text/0000-macro-reform.md @@ -22,35 +22,86 @@ edges](https://github.com/rust-lang/rfcs/issues/440). A few of the big ones: - There's no namespacing at all - You can't control which macros are imported from a crate - You need the feature-gated `#[phase(plugin)]` to import macros +- It's confusing that macro definition is itself a macro invocation, with a side effect + on the expansion context These issues in particular are things we have a chance of addressing for 1.0. This RFC contains plans to do so. # Detailed design -## Rename +## `macro` items -Rename `macro_rules!` to `macro!`, analogous to `fn`, `struct`, etc. +Introduce a new keyword `macro`. Use it for a new kind of item, a macro definition: -## `#[visible(...)]` +```rust +// first example from the Macros Guide +macro early_return { + ($inp:expr $sp:ident) => ( + match $inp { + $sp(x) => { return x; } + _ => {} + } + ) +} +``` + +`macro_rules!` already allows `{ }` for the macro body, but the convention is +`( )` for some reason. In accepting this RFC we would change to a `{ }` +convention for consistency with the rest of the language. -`#[macro_export]` and `#[macro_escape]` are replaced with a new attribute -applied to a `macro!` invocation: +There are two scope qualifiers for `macro`: -* `#[visible(this_crate)]` — this macro "escapes" up the module hierarchy to - the crate root, so it can be used anywhere in the crate after the definition - (according to a depth-first traversal). This is like putting `#[macro_escape]` - on the module and all its ancestors, but applies *only* to the macro with the - `#[visible]` attribute. +* `pub` — this macro "escapes" up the module hierarchy to the crate root, so it + can be used anywhere in this crate after its definition (according to a + depth-first traversal). This is like putting `#[macro_escape]` on the module + and all its ancestors, but applies *only* to the macro with the `#[visible]` + attribute. -* `#[visible(other_crates)]` — same meaning as `#[macro_export]` today +* `extern` — this macro can be imported by other crates, i.e. the same meaning + as `#[macro_export]` today -These can be combined as `#[visible(this_crate, other_crates)]` (in either -order). +These can be used together. The default (as today) is that the macro is only visible within the lexical scope where it is defined. +The `Item` AST node changes as follows: + +```rust +pub enum Item_ { + // ... + + /// A macro definition. + ItemDefineMacro(Vec), + + /// A macro invocation. + ItemUseMacro(Mac), // was called ItemMac +} +``` + +While it's unfortunate that AST types can't represent a `macro` item as +anything richer than a token tree, this is not a regression from the status quo +with `macro_rules!`, which parses as an `ItemMac`. + +This also provides flexibility to make backwards-compatible changes. One can +imagine + +```rust +#[procedural] macro match_token { + fn expand(cx: &mut ExtCtxt, span: Span, toks: &[TokenTree]) + -> Box { + // ... + } +} +``` + +though working out the details is far outside the scope of this RFC. + +We are free to change the AST and parsing after 1.0 as long as the old syntax +still works. So we could have a separate enum varient for procedural macros +parsed as function decls. + ## `$crate` Add a special metavar `$crate` which expands to `::foo` when the macro was @@ -65,47 +116,48 @@ rebase. ## Crate scope for macros Instead of a single global namespace for macro definitions, we now have one -namespace per crate. We introduce an attribute to `use` macros with optional -renaming: +namespace per crate. We introduce a new view item, `use macro`: ```rust -#[use_macros(std::vec, std::panic as fail)] +use macro std::vec; +use macro std::panic as fail; ``` -(We'd need to extend attribute syntax, or change this to be compatible.) - -This can be applied to a module, a function, or anywhere else attributes are -allowed. - There's no way to invoke a macro with a qualified name; this obviates the need -for changes to the grammar / parser. +to change the parsing of expressions, patterns, etc. -This isn't an actual `use` item because macro expansion happens before name -resolution, and libsyntax knows nothing about the latter. +The `macro` keyword is important because it signals that this is something +different from the usual name resolution. `use macro` is a memorable and +searchable name for the feature. -This change applies to `macro!` and to all other syntax extensions, even -decorators that are used as attributes. +`use macro` can be qualified with `pub` to get the same "escape" behavior as +`pub macro`. `use macro` also allows globs. For example: + +```rust +#![no_std] +extern crate core; +pub use macro core::*; +``` Many macros expand using other "private macros" as an implementation detail. For example, librustc's `declare_lint!` uses `lint_initializer!`. The client should not know about this macro, although it still needs to be exported for -cross-crate use. The solution is to allow `use_macros` on `macro!` itself, and -allow `$crate` in that context: +cross-crate use. For this reason we allow `use macro` within a macro +definition, and allow `$crate` in that context. ```rust /// Not to be imported directly. -#[visible(other_crates)] -macro! lint_initializer ( ... ) +extern macro lint_initializer { ... } /// Declare a lint. -#[visible(other_crates)] -#[use_macros($crate::lint_initializer)] -macro! declare_lint ( +extern macro declare_lint { + use macro $crate::lint_initializer; + ($name:ident, $level:ident, $desc:expr) => ( static $name: &'static $crate::lint::Lint = &lint_initializer!($name, $level, $desc); ) -) +} ``` The macro `lint_initializer!` will be visible only during further expansion of @@ -113,31 +165,29 @@ the result of invoking `declare_lint!`. Procedural macros need their own way to manipulate the expansion context, but that's an unstable internal API, so it's outside the scope of this RFC. -Ideally the implementation of `macro!` will use the same API. ## Macro re-export With `$crate` we can easily re-export macros that were imported from another -crate. +crate. This is accomplished with `extern use macro`. For example, libcollections defines a `vec!` macro, which would now look like: ```rust -#[visible(other_crates)] -macro! vec ( +extern macro vec { ($($e:expr),*) => ({ let mut _temp = $crate::vec::Vec::new(); $(_temp.push($e);)* _temp }) -) +} ``` Currently, libstd duplicates this macro in its own `macros.rs`. Now it could do ```rust -#![reexport_macros(collections::vec)] +extern use macro collections::vec; ``` as long as the module `std::vec` is interface-compatible with @@ -168,11 +218,11 @@ make sure our design is compatible. This is big churn on a major feature, not long before 1.0. -We can ship improved versions of `macro!` in a back-compat way (in theory; I -would like to smoke test this idea before 1.0). So we could defer all this -reform until after 1.0. The main reason not to is macro import/export. Right -now every macro you import will be expanded using your local copy of `macro!`, -regardless of what the macro author had in mind. +We can ship improved versions of `macro_rules!` in a back-compat way (in +theory; I would like to smoke test this idea before 1.0). So we could defer +much of this reform until after 1.0. The main reason not to is macro +import/export. Right now every macro you import will be expanded using your +local copy of `macro_rules!`, regardless of what the macro author had in mind. # Alternatives @@ -202,15 +252,32 @@ reform. Two ways this could work out: # Unresolved questions -Do we support globs in `use_macros`? How does prelude injection work? -How do I get all of libcore's macros if my crate is `no_std`? +Should we forbid `$crate` in non-`extern` macros? It seems useless, however I +think we should allow it anyway, to encourage the habit of writing `$crate::` +for any references to the local crate. -Does `use_macros` also define the name in child modules? In a sense this is the -more natural thing to do in libsyntax, but it's inconsistent with normal `use` -items. +With `macro_rules!` gone, do we eliminate `IdentTT`-style macro syntax? There +are some out-of-tree users. Maybe macros in item position should automatically +look for an identifier, desugaring an invocation like + +```rust +foo! name { + ... +} +``` + +into + +```rust +foo! { name { + ... +} } +``` -All of the syntax is bikeshedable. For example, should `use_macros` include -the exclamation point? What about when it applies to item decorator attributes? +This is appropriate in item position because every non-macro item defines one +name. However macros can expand to multiple items, so we should keep both +forms. -Allowing `$crate` in attributes is weird. Maybe we should use some other -syntax that fits with existing attribute parsing. +This would make the syntax accessible from non-procedural macros, and would cut +down on special cases elsewhere in the compiler. This change would be a +separate RFC though. From 193e2ab7ad5a11bc9611b0fc5792832249eeb4df Mon Sep 17 00:00:00 2001 From: Keegan McAllister Date: Wed, 12 Nov 2014 16:17:56 -0800 Subject: [PATCH 04/27] Add item macro sugar --- text/0000-macro-reform.md | 99 +++++++++++++++++++++++++++++---------- 1 file changed, 73 insertions(+), 26 deletions(-) diff --git a/text/0000-macro-reform.md b/text/0000-macro-reform.md index d9c07c6177f..27dd7a5f623 100644 --- a/text/0000-macro-reform.md +++ b/text/0000-macro-reform.md @@ -166,6 +166,79 @@ the result of invoking `declare_lint!`. Procedural macros need their own way to manipulate the expansion context, but that's an unstable internal API, so it's outside the scope of this RFC. +## Item macro sugar + +An item defines one name in the current module, and can have "adjective" +qualifiers such as `pub`, `extern`, etc. We extend macro invocation in item +position to reflect this form. The new invocation syntax is + +```rust + foo! { + +} +``` + +where `` is a sequence of zero or more keywords from the set `pub` +`priv` `extern` `unsafe` `const` `static` `box` `ref` `mut`. This list is +pretty arbitrary but can be expanded later. For now it only includes keywords +that have existing meaning(s) in Rust, which should keep things somewhat under +control. + +This form of item macro is a bit like defining your own keyword that can take +the place of `fn`, `struct`, etc. + +We keep the existing syntax for invoking macros in item position, and desugar +the above syntax to it, as + +```rust +foo!( : { }) +``` + +This extends the existing `IdentTT` macro form and makes it available to +non-procedural macros, while reducing special cases within the compiler. + +An item macro invocation in either the sugared or unsugared form may expand to +zero or more items. + +Item macro sugar is an integral part of macro reform because it's needed to +make `pub` and `extern` work with macro-defining macros. For example + +```rust +macro mega_macro { + ($($qual:ident)* : $name:ident $body:tt) => ( + $($qual)* macro $name { + // ... + } + ) +} +``` + +can be invoked as + +```rust +pub extern mega_macro! foo { + (bar) => (3u) +} +``` + +Here `mega_macro!` takes the place of the built-in `macro` keyword. + +The applications go beyond macro-defining macros. For example, [this macro in +rustc](https://github.com/rust-lang/rust/blob/221fc1e3cdcc208e1bb7debcc2de27d47c847747/src/librustc/lint/mod.rs#L83-L95) +could change to support invocations like + +```rust +lint! UNUSED_ATTRIBUTES { + Warn, "detects attributes that were not used by the compiler" +} + +pub lint! PATH_STATEMENTS { + Warn, "path statements with no effect" +} +``` + +which is considerably nicer than the current syntax. + ## Macro re-export With `$crate` we can easily re-export macros that were imported from another @@ -255,29 +328,3 @@ reform. Two ways this could work out: Should we forbid `$crate` in non-`extern` macros? It seems useless, however I think we should allow it anyway, to encourage the habit of writing `$crate::` for any references to the local crate. - -With `macro_rules!` gone, do we eliminate `IdentTT`-style macro syntax? There -are some out-of-tree users. Maybe macros in item position should automatically -look for an identifier, desugaring an invocation like - -```rust -foo! name { - ... -} -``` - -into - -```rust -foo! { name { - ... -} } -``` - -This is appropriate in item position because every non-macro item defines one -name. However macros can expand to multiple items, so we should keep both -forms. - -This would make the syntax accessible from non-procedural macros, and would cut -down on special cases elsewhere in the compiler. This change would be a -separate RFC though. From 82f7c162272066cf422c8a98b192d85deb633f5c Mon Sep 17 00:00:00 2001 From: Keegan McAllister Date: Wed, 12 Nov 2014 16:28:26 -0800 Subject: [PATCH 05/27] Unspecify order of procedural macro side effects --- text/0000-macro-reform.md | 17 +++++++++++++++++ 1 file changed, 17 insertions(+) diff --git a/text/0000-macro-reform.md b/text/0000-macro-reform.md index 27dd7a5f623..b7ddd20117a 100644 --- a/text/0000-macro-reform.md +++ b/text/0000-macro-reform.md @@ -286,6 +286,23 @@ these "arguments" are available in the registrar and can be used to configure how the plugin works. This RFC does not cover that feature; I just want to make sure our design is compatible. +## Unspecify order of procedural macro side effects + +We clarify that the ordering of expansion, hence side effects, for separate +(i.e. non-nested) procedural macro invocations is unspecified. This does not +affect the stable language, because procedural macros are not part of it, and +expansion of pattern-based macros cannot have side effects. + +Interacting side effects between procedural macros is messy in general. It's +much better for a macro to approximate a pure function of its input, plus an +"environment" that does not change during macro expansion. I claim that this +is usually possible (see +[discussion](https://github.com/rust-lang/rfcs/pull/453#issuecomment-62813856)). + +The main reason to consider this now in an RFC is to make sure that built-in +procedural macros exposed to stable code can comply. Reserving the right to +change expansion order allows us to pursue more sophisticated approaches to the +name resolution problem after 1.0. # Drawbacks From 3ad1ce363e8e0daa211a978d889bcf68e6d4ee6c Mon Sep 17 00:00:00 2001 From: Keegan McAllister Date: Wed, 12 Nov 2014 16:29:36 -0800 Subject: [PATCH 06/27] Add note about $crate lint --- text/0000-macro-reform.md | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/text/0000-macro-reform.md b/text/0000-macro-reform.md index b7ddd20117a..d0035ad2faf 100644 --- a/text/0000-macro-reform.md +++ b/text/0000-macro-reform.md @@ -113,6 +113,10 @@ enables macro re-export (see below). It is [implemented and tested](https://github.com/kmcallister/rust/commits/macro-reexport) but needs a rebase. +Add a lint to warn in cases where an `extern` macro has paths that are not +absolute-with-crate or `$crate`-relative. This will have some (hopefully rare) +false positives, and is not fully fleshed out yet. + ## Crate scope for macros Instead of a single global namespace for macro definitions, we now have one From df4bb87ba03ce0161ba00760f735ac36532d2897 Mon Sep 17 00:00:00 2001 From: Keegan McAllister Date: Wed, 12 Nov 2014 16:34:09 -0800 Subject: [PATCH 07/27] Spellcheck --- text/0000-macro-reform.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/text/0000-macro-reform.md b/text/0000-macro-reform.md index d0035ad2faf..0bc8764fe74 100644 --- a/text/0000-macro-reform.md +++ b/text/0000-macro-reform.md @@ -99,7 +99,7 @@ imagine though working out the details is far outside the scope of this RFC. We are free to change the AST and parsing after 1.0 as long as the old syntax -still works. So we could have a separate enum varient for procedural macros +still works. So we could have a separate enum variant for procedural macros parsed as function decls. ## `$crate` @@ -312,7 +312,7 @@ name resolution problem after 1.0. This is big churn on a major feature, not long before 1.0. -We can ship improved versions of `macro_rules!` in a back-compat way (in +We can ship improved versions of `macro_rules!` in a back-compatible way (in theory; I would like to smoke test this idea before 1.0). So we could defer much of this reform until after 1.0. The main reason not to is macro import/export. Right now every macro you import will be expanded using your From 2ef9932cb6cb8e7c35f7ad2d62fdc0ed88e3ae8b Mon Sep 17 00:00:00 2001 From: Keegan McAllister Date: Wed, 12 Nov 2014 17:35:05 -0800 Subject: [PATCH 08/27] Add acknowledgements --- text/0000-macro-reform.md | 13 +++++++++++++ 1 file changed, 13 insertions(+) diff --git a/text/0000-macro-reform.md b/text/0000-macro-reform.md index 0bc8764fe74..b42e61bae5e 100644 --- a/text/0000-macro-reform.md +++ b/text/0000-macro-reform.md @@ -349,3 +349,16 @@ reform. Two ways this could work out: Should we forbid `$crate` in non-`extern` macros? It seems useless, however I think we should allow it anyway, to encourage the habit of writing `$crate::` for any references to the local crate. + +# Acknowledgements + +This proposal is edited by Keegan McAllister. It has been refined through many +engaging discussions with: + +* Brian Anderson, Lars Bergstrom, Nick Cameron, John Clements, Alex Crichton, Cathy Douglass, Steven Fackler, Manish Goregaokar, Dave Herman, Steve Klabnik, Felix S. Klock II, Niko Matsakis, Paul Stansifer, Sam Tobin-Hochstadt, Aaron Turon, Huon Wilson, Brendan Zabarauskas, Cameron Zwarich +* *GitHub*: `@bill-myers` `@blaenk` `@comex` `@glaebhoerl` `@mitchmindtree` `@mitsuhiko` `@P1Start` `@petrochenkov` `@skinner` +* *Reddit*: `ippa` `Quxxy` `rime-frost` `Sinistersnare` `tejp` +* *IRC*: `bstrie` `ChrisMorgan` `cmr` `Earnestly` `eddyb` `tiffany` + +My apologies if I've forgotten you, used an un-preferred name, or accidentally +categorized you as several different people. Pull requests are welcome :) From 01563f6cf836c5adee14192d8ee5907f13f0491d Mon Sep 17 00:00:00 2001 From: Keegan McAllister Date: Wed, 12 Nov 2014 18:00:17 -0800 Subject: [PATCH 09/27] Demonstrate alternative procedural macro sugar --- text/0000-macro-reform.md | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/text/0000-macro-reform.md b/text/0000-macro-reform.md index b42e61bae5e..cf594235deb 100644 --- a/text/0000-macro-reform.md +++ b/text/0000-macro-reform.md @@ -96,6 +96,16 @@ imagine } ``` +or + +```rust +macro atom { + ($name:tt) => fn expand(...) { + // ... + } +} +``` + though working out the details is far outside the scope of this RFC. We are free to change the AST and parsing after 1.0 as long as the old syntax From 1fc4d1e6bf80518621d77aae2225f0d865b5aef0 Mon Sep 17 00:00:00 2001 From: Keegan McAllister Date: Wed, 12 Nov 2014 18:24:52 -0800 Subject: [PATCH 10/27] Add a note about deprecating use macro --- text/0000-macro-reform.md | 14 ++++++++++++++ 1 file changed, 14 insertions(+) diff --git a/text/0000-macro-reform.md b/text/0000-macro-reform.md index cf594235deb..8b4db5c0bda 100644 --- a/text/0000-macro-reform.md +++ b/text/0000-macro-reform.md @@ -180,6 +180,20 @@ the result of invoking `declare_lint!`. Procedural macros need their own way to manipulate the expansion context, but that's an unstable internal API, so it's outside the scope of this RFC. +In the long run, + +```rust +use macro std::vec; +``` + +may end up as a deprecated synonym for + +```rust +use std::vec!; +``` + +but maintaining this synonym does not seem like a large burden. + ## Item macro sugar An item defines one name in the current module, and can have "adjective" From f517cd1176dbfd1ec4455db8fd48f06325bc7cab Mon Sep 17 00:00:00 2001 From: Keegan McAllister Date: Wed, 12 Nov 2014 18:25:04 -0800 Subject: [PATCH 11/27] Add lazy-static.rs as an example --- text/0000-macro-reform.md | 11 ++++++++++- 1 file changed, 10 insertions(+), 1 deletion(-) diff --git a/text/0000-macro-reform.md b/text/0000-macro-reform.md index 8b4db5c0bda..310f758cf20 100644 --- a/text/0000-macro-reform.md +++ b/text/0000-macro-reform.md @@ -266,6 +266,15 @@ pub lint! PATH_STATEMENTS { ``` which is considerably nicer than the current syntax. +[`lazy-static.rs`](https://github.com/Kimundi/lazy-static.rs) could support +syntax like + +```rust +pub lazy_static! LOG { + : Mutex> + = Mutex::new(Vec::with_capacity(50_000)) +} +``` ## Macro re-export @@ -380,7 +389,7 @@ This proposal is edited by Keegan McAllister. It has been refined through many engaging discussions with: * Brian Anderson, Lars Bergstrom, Nick Cameron, John Clements, Alex Crichton, Cathy Douglass, Steven Fackler, Manish Goregaokar, Dave Herman, Steve Klabnik, Felix S. Klock II, Niko Matsakis, Paul Stansifer, Sam Tobin-Hochstadt, Aaron Turon, Huon Wilson, Brendan Zabarauskas, Cameron Zwarich -* *GitHub*: `@bill-myers` `@blaenk` `@comex` `@glaebhoerl` `@mitchmindtree` `@mitsuhiko` `@P1Start` `@petrochenkov` `@skinner` +* *GitHub*: `@bill-myers` `@blaenk` `@comex` `@glaebhoerl` `@Kimundi` `@mitchmindtree` `@mitsuhiko` `@P1Start` `@petrochenkov` `@skinner` * *Reddit*: `ippa` `Quxxy` `rime-frost` `Sinistersnare` `tejp` * *IRC*: `bstrie` `ChrisMorgan` `cmr` `Earnestly` `eddyb` `tiffany` From fea0c6d5270c22549b4a8a36df93b59f191171a5 Mon Sep 17 00:00:00 2001 From: Keegan McAllister Date: Wed, 12 Nov 2014 19:07:49 -0800 Subject: [PATCH 12/27] Add table of contents for design --- text/0000-macro-reform.md | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/text/0000-macro-reform.md b/text/0000-macro-reform.md index 310f758cf20..dafaa3b6f54 100644 --- a/text/0000-macro-reform.md +++ b/text/0000-macro-reform.md @@ -30,6 +30,16 @@ This RFC contains plans to do so. # Detailed design +Skip ahead to + +* [`macro` items](#macro-items) +* [`$crate`](#crate) +* [Crate scope for macros](#crate-scope-for-macros) +* [Item macro sugar](#item-macro-sugar) +* [Macro re-export](#macro-re-export) +* [Auto-load macros](#auto-load-macros) +* [Unspecify order of procedural macro side effects](#unspecify-order-of-procedural-macro-side-effects) + ## `macro` items Introduce a new keyword `macro`. Use it for a new kind of item, a macro definition: From 3dd2ee9da717d9b4c79a9984928c8050da2c29c0 Mon Sep 17 00:00:00 2001 From: Keegan McAllister Date: Wed, 12 Nov 2014 19:23:24 -0800 Subject: [PATCH 13/27] Add #[plugin(... args ...)] --- text/0000-macro-reform.md | 18 ++++++++++++------ 1 file changed, 12 insertions(+), 6 deletions(-) diff --git a/text/0000-macro-reform.md b/text/0000-macro-reform.md index dafaa3b6f54..e2c28a48f2f 100644 --- a/text/0000-macro-reform.md +++ b/text/0000-macro-reform.md @@ -37,7 +37,7 @@ Skip ahead to * [Crate scope for macros](#crate-scope-for-macros) * [Item macro sugar](#item-macro-sugar) * [Macro re-export](#macro-re-export) -* [Auto-load macros](#auto-load-macros) +* [`#[plugin]` attribute](#plugin-attribute) * [Unspecify order of procedural macro side effects](#unspecify-order-of-procedural-macro-side-effects) ## `macro` items @@ -316,7 +316,7 @@ as long as the module `std::vec` is interface-compatible with (Actually the current libstd `vec!` is completely different for efficiency, but it's just an example.) -## Auto-load macros +## `#[plugin]` attribute Since macros are now crate-scoped, we can load macros from every `extern crate` without a special attribute. (Probably we should exclude `extern crate`s that @@ -328,10 +328,16 @@ only controls whether to search for and run a plugin registrar function. The plugin itself will decide whether it's to be linked at runtime, by calling a `Registry` method. -In the future I would like to support `#[plugin(... any metas ...)]` where -these "arguments" are available in the registrar and can be used to configure -how the plugin works. This RFC does not cover that feature; I just want to -make sure our design is compatible. +`#[plugin]` takes an optional "arguments list" of the form + +```rust +#[plugin(foo="bar", ... any metas ...)] +extern crate myplugin; +``` + +rustc itself will not interpret these attribute [meta +items](http://doc.rust-lang.org/syntax/ast/enum.MetaItem_.html), but will make +them available to the plugin through a `Registry` method. ## Unspecify order of procedural macro side effects From 6f7ea0c46d5474cb49c2ae46cc6ffa28780247e6 Mon Sep 17 00:00:00 2001 From: Keegan McAllister Date: Wed, 12 Nov 2014 19:35:12 -0800 Subject: [PATCH 14/27] Add more acknowledgements --- text/0000-macro-reform.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/text/0000-macro-reform.md b/text/0000-macro-reform.md index e2c28a48f2f..3029da0a699 100644 --- a/text/0000-macro-reform.md +++ b/text/0000-macro-reform.md @@ -404,9 +404,9 @@ for any references to the local crate. This proposal is edited by Keegan McAllister. It has been refined through many engaging discussions with: -* Brian Anderson, Lars Bergstrom, Nick Cameron, John Clements, Alex Crichton, Cathy Douglass, Steven Fackler, Manish Goregaokar, Dave Herman, Steve Klabnik, Felix S. Klock II, Niko Matsakis, Paul Stansifer, Sam Tobin-Hochstadt, Aaron Turon, Huon Wilson, Brendan Zabarauskas, Cameron Zwarich +* Brian Anderson, Shachaf Ben-Kiki, Lars Bergstrom, Nick Cameron, John Clements, Alex Crichton, Cathy Douglass, Steven Fackler, Manish Goregaokar, Dave Herman, Steve Klabnik, Felix S. Klock II, Niko Matsakis, Matthew McPherrin, Paul Stansifer, Sam Tobin-Hochstadt, Aaron Turon, Huon Wilson, Brendan Zabarauskas, Cameron Zwarich * *GitHub*: `@bill-myers` `@blaenk` `@comex` `@glaebhoerl` `@Kimundi` `@mitchmindtree` `@mitsuhiko` `@P1Start` `@petrochenkov` `@skinner` -* *Reddit*: `ippa` `Quxxy` `rime-frost` `Sinistersnare` `tejp` +* *Reddit*: `ippa` `Mystor` `Quxxy` `rime-frost` `Sinistersnare` `tejp` * *IRC*: `bstrie` `ChrisMorgan` `cmr` `Earnestly` `eddyb` `tiffany` My apologies if I've forgotten you, used an un-preferred name, or accidentally From 9bbb13fb686efed4ef46c654d82711238326081f Mon Sep 17 00:00:00 2001 From: Keegan McAllister Date: Wed, 12 Nov 2014 21:59:10 -0800 Subject: [PATCH 15/27] Update for new syntax --- text/0000-macro-reform.md | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/text/0000-macro-reform.md b/text/0000-macro-reform.md index 3029da0a699..77daccd04a4 100644 --- a/text/0000-macro-reform.md +++ b/text/0000-macro-reform.md @@ -65,8 +65,7 @@ There are two scope qualifiers for `macro`: * `pub` — this macro "escapes" up the module hierarchy to the crate root, so it can be used anywhere in this crate after its definition (according to a depth-first traversal). This is like putting `#[macro_escape]` on the module - and all its ancestors, but applies *only* to the macro with the `#[visible]` - attribute. + and all its ancestors, but applies *only* to the macro with `pub`. * `extern` — this macro can be imported by other crates, i.e. the same meaning as `#[macro_export]` today From ef13bdc87587ec1d1bb8a69f07089cb8ea9af3c6 Mon Sep 17 00:00:00 2001 From: Keegan McAllister Date: Sun, 16 Nov 2014 21:46:47 -0800 Subject: [PATCH 16/27] Reorganize and clarify --- text/0000-macro-reform.md | 375 ++++++++++++++++++++++---------------- 1 file changed, 213 insertions(+), 162 deletions(-) diff --git a/text/0000-macro-reform.md b/text/0000-macro-reform.md index 77daccd04a4..8d2840545ad 100644 --- a/text/0000-macro-reform.md +++ b/text/0000-macro-reform.md @@ -4,7 +4,7 @@ # Summary -Various enhancements to `macro_rules!` ahead of its standardization in 1.0. +Various enhancements to macros ahead of their standardization in 1.0. **Note**: This is not the final Rust macro system design for all time. Rather, it addresses the largest usability problems within the limited time frame for @@ -23,107 +23,80 @@ edges](https://github.com/rust-lang/rfcs/issues/440). A few of the big ones: - You can't control which macros are imported from a crate - You need the feature-gated `#[phase(plugin)]` to import macros - It's confusing that macro definition is itself a macro invocation, with a side effect - on the expansion context + on the syntax environment These issues in particular are things we have a chance of addressing for 1.0. This RFC contains plans to do so. -# Detailed design +# Semantic changes -Skip ahead to +These are the substantial changes to the macro system. The examples also use +the improved syntax, described later. -* [`macro` items](#macro-items) -* [`$crate`](#crate) -* [Crate scope for macros](#crate-scope-for-macros) -* [Item macro sugar](#item-macro-sugar) -* [Macro re-export](#macro-re-export) -* [`#[plugin]` attribute](#plugin-attribute) -* [Unspecify order of procedural macro side effects](#unspecify-order-of-procedural-macro-side-effects) +## Crate scope for macros -## `macro` items +In this document, the "syntax environment" refers to the set of syntax +extensions that can be invoked at a given position in the crate. The names in +the syntax environment are simple unqualified identifiers such as `panic` and +`vec`. Informally we may write `vec!` to distinguish from an ordinary item. +However, the exclamation point is really part of the invocation syntax, not the +name, and some syntax extensions are invoked with no exclamation point, for +example item decorators like `deriving`. -Introduce a new keyword `macro`. Use it for a new kind of item, a macro definition: +The first proposed change is that macros imported from another crate will not +automatically end up in the syntax environment, as they do today. Instead, you +can bring an imported macro into the syntax environment with a new view item, +`use macro`: ```rust -// first example from the Macros Guide -macro early_return { - ($inp:expr $sp:ident) => ( - match $inp { - $sp(x) => { return x; } - _ => {} - } - ) -} +use macro std::vec; +use macro std::panic as fail; +use macro core::*; ``` -`macro_rules!` already allows `{ }` for the macro body, but the convention is -`( )` for some reason. In accepting this RFC we would change to a `{ }` -convention for consistency with the rest of the language. - -There are two scope qualifiers for `macro`: - -* `pub` — this macro "escapes" up the module hierarchy to the crate root, so it - can be used anywhere in this crate after its definition (according to a - depth-first traversal). This is like putting `#[macro_escape]` on the module - and all its ancestors, but applies *only* to the macro with `pub`. - -* `extern` — this macro can be imported by other crates, i.e. the same meaning - as `#[macro_export]` today - -These can be used together. - -The default (as today) is that the macro is only visible within the lexical -scope where it is defined. - -The `Item` AST node changes as follows: - -```rust -pub enum Item_ { - // ... - - /// A macro definition. - ItemDefineMacro(Vec), +The syntax environment still consists of unqualified names. There's no way to +invoke a macro through a qualified name. This obviates the need to change the +parsing of expressions, patterns, etc. - /// A macro invocation. - ItemUseMacro(Mac), // was called ItemMac -} -``` +The `macro` keyword is important because it signals that this is something +different from the usual name resolution. `use macro` is a memorable and +searchable name for the feature. -While it's unfortunate that AST types can't represent a `macro` item as -anything richer than a token tree, this is not a regression from the status quo -with `macro_rules!`, which parses as an `ItemMac`. +The `use macro` view item only affects the syntax environment of the block or +module where it appears, and only from the point of appearance onward. Unlike +a normal `use` item, this includes child modules (in the same file or others). -This also provides flexibility to make backwards-compatible changes. One can -imagine +Many macros expand using other "private macros" as an implementation detail. +For example, librustc's `declare_lint!` uses `lint_initializer!`. The client +should not know about this macro, although it still needs to be exported for +cross-crate use. For this reason we allow `use macro` within a macro +definition. ```rust -#[procedural] macro match_token { - fn expand(cx: &mut ExtCtxt, span: Span, toks: &[TokenTree]) - -> Box { - // ... - } -} -``` +/// Not to be imported directly. +extern macro lint_initializer { ... } -or +/// Declare a lint. +extern macro declare_lint { + // See below for $crate + use macro $crate::lint_initializer; -```rust -macro atom { - ($name:tt) => fn expand(...) { - // ... - } + ($name:ident, $level:ident, $desc:expr) => ( + static $name: &'static $crate::lint::Lint + = &lint_initializer!($name, $level, $desc); + ) } ``` -though working out the details is far outside the scope of this RFC. +The macro `lint_initializer!` will be visible only during further expansion of +the result of invoking `declare_lint!`. -We are free to change the AST and parsing after 1.0 as long as the old syntax -still works. So we could have a separate enum variant for procedural macros -parsed as function decls. +Procedural macros need their own way to manipulate the syntax environment, but +that's an unstable internal API, so it's outside the scope of this RFC. ## `$crate` -Add a special metavar `$crate` which expands to `::foo` when the macro was +We add a special metavar `$crate` which expands to `::foo` when a macro was imported from crate `foo`, and to nothing when it was defined in-crate. `$crate::bar::baz` will be an absolute path either way. @@ -132,82 +105,88 @@ enables macro re-export (see below). It is [implemented and tested](https://github.com/kmcallister/rust/commits/macro-reexport) but needs a rebase. -Add a lint to warn in cases where an `extern` macro has paths that are not -absolute-with-crate or `$crate`-relative. This will have some (hopefully rare) -false positives, and is not fully fleshed out yet. +# New syntax -## Crate scope for macros +We also clean up macro syntax in a way that complements the semantic changes above. -Instead of a single global namespace for macro definitions, we now have one -namespace per crate. We introduce a new view item, `use macro`: +## `macro_rules!` becomes `macro` + +The new macro definition syntax uses the `macro` keyword introduced above: ```rust -use macro std::vec; -use macro std::panic as fail; +// first example from the Macros Guide +macro early_return { + ($inp:expr $sp:ident) => ( + match $inp { + $sp(x) => { return x; } + _ => {} + } + ) +} ``` -There's no way to invoke a macro with a qualified name; this obviates the need -to change the parsing of expressions, patterns, etc. +This is an ordinary item, like `fn` or `struct`, not a macro invocation. It +defines a new macro in the syntax environment of the enclosing block or module. -The `macro` keyword is important because it signals that this is something -different from the usual name resolution. `use macro` is a memorable and -searchable name for the feature. - -`use macro` can be qualified with `pub` to get the same "escape" behavior as -`pub macro`. `use macro` also allows globs. For example: +The new macro can be used immediately. There is no way to `use macro` a macro +defined in the same crate, and no need to do so. -```rust -#![no_std] -extern crate core; -pub use macro core::*; -``` - -Many macros expand using other "private macros" as an implementation detail. -For example, librustc's `declare_lint!` uses `lint_initializer!`. The client -should not know about this macro, although it still needs to be exported for -cross-crate use. For this reason we allow `use macro` within a macro -definition, and allow `$crate` in that context. +If qualified by `pub`, the macro escapes the syntax environment for the +enclosing block/module and becomes available throughout the rest of the crate +(according to depth-first search). This is like putting `#[macro_escape]` on +the module and all its ancestors, but applies *only* to the macro with `pub`. -```rust -/// Not to be imported directly. -extern macro lint_initializer { ... } +`macro_rules!` already allows `{ }` for the macro body, but the convention is +`( )` for some reason. In accepting this RFC we would change to a `{ }` +convention for consistency with the rest of the language. -/// Declare a lint. -extern macro declare_lint { - use macro $crate::lint_initializer; +## Macro export and re-export - ($name:ident, $level:ident, $desc:expr) => ( - static $name: &'static $crate::lint::Lint - = &lint_initializer!($name, $level, $desc); - ) -} -``` +A `macro` item qualified by `extern` becomes available to other crates. That +is, it can be the target of `use macro`. Or put another way, `extern macro` +works the way `#[macro_export] macro_rules!` does today. Adding `extern` has +no effect on the syntax environment for the current crate. -The macro `lint_initializer!` will be visible only during further expansion of -the result of invoking `declare_lint!`. +`pub` and `extern` may be used together on the same `macro` definition, since +their effects are independent. -Procedural macros need their own way to manipulate the expansion context, but -that's an unstable internal API, so it's outside the scope of this RFC. +We can also re-export macros that were imported from another crate. This is +accomplished with `extern use macro`. -In the long run, +For example, libcollections defines a `vec!` macro, which would now look like: ```rust -use macro std::vec; +extern macro vec { + ($($e:expr),*) => ({ + let mut _temp = $crate::vec::Vec::new(); + $(_temp.push($e);)* + _temp + }) +} ``` -may end up as a deprecated synonym for +Currently, libstd duplicates this macro in its own `macros.rs`. Now it could +do ```rust -use std::vec!; +extern use macro collections::vec; ``` -but maintaining this synonym does not seem like a large burden. +as long as the module `std::vec` is interface-compatible with +`collections::vec`. + +(Actually the current libstd `vec!` is completely different for efficiency, but +it's just an example.) + +Because macros are exported in crate metadata as strings, macro re-export "just +works" as soon as `$crate` is available. It's implemented as part of the +`$crate` branch mentioned above. ## Item macro sugar -An item defines one name in the current module, and can have "adjective" -qualifiers such as `pub`, `extern`, etc. We extend macro invocation in item -position to reflect this form. The new invocation syntax is +In Rust's syntax, an item defines one name in the current module, and can have +"adjective" qualifiers such as `pub`, `extern`, etc. We extend macro +invocation in item position to reflect this form. The new invocation syntax is ```rust foo! { @@ -285,49 +264,26 @@ pub lazy_static! LOG { } ``` -## Macro re-export - -With `$crate` we can easily re-export macros that were imported from another -crate. This is accomplished with `extern use macro`. - -For example, libcollections defines a `vec!` macro, which would now look like: - -```rust -extern macro vec { - ($($e:expr),*) => ({ - let mut _temp = $crate::vec::Vec::new(); - $(_temp.push($e);)* - _temp - }) -} -``` - -Currently, libstd duplicates this macro in its own `macros.rs`. Now it could -do - -```rust -extern use macro collections::vec; -``` - -as long as the module `std::vec` is interface-compatible with -`collections::vec`. - -(Actually the current libstd `vec!` is completely different for efficiency, but -it's just an example.) - ## `#[plugin]` attribute -Since macros are now crate-scoped, we can load macros from every `extern crate` -without a special attribute. (Probably we should exclude `extern crate`s that -aren't at the crate root, because there's no way `$crate` paths will be -correct.) +Since macros no longer automatically pollute the syntax environment, we can +load them from every `extern crate`. (Probably we should exclude +`extern crate`s that aren't at the crate root, because there's no way `$crate` +paths will be correct.) `#[phase(plugin)]` becomes simply `#[plugin]` and is still feature-gated. It only controls whether to search for and run a plugin registrar function. The plugin itself will decide whether it's to be linked at runtime, by calling a `Registry` method. -`#[plugin]` takes an optional "arguments list" of the form +# Other improvements + +These are somewhat related to the above, but could be spun off into separate +RFCs. + +## Arguments to `#[plugin]` + +`#[plugin]` will take an optional "arguments list" of the form ```rust #[plugin(foo="bar", ... any metas ...)] @@ -338,8 +294,15 @@ rustc itself will not interpret these attribute [meta items](http://doc.rust-lang.org/syntax/ast/enum.MetaItem_.html), but will make them available to the plugin through a `Registry` method. +This facilitates plugin configuration. The alternative in many cases is to use +interacting side effects between procedural macros, which are harder to reason +about. + ## Unspecify order of procedural macro side effects +This is basically an internal-only change, but I mention it in this RFC (for +now) because it was discussed alongside the rest of this. + We clarify that the ordering of expansion, hence side effects, for separate (i.e. non-nested) procedural macro invocations is unspecified. This does not affect the stable language, because procedural macros are not part of it, and @@ -356,6 +319,88 @@ procedural macros exposed to stable code can comply. Reserving the right to change expansion order allows us to pursue more sophisticated approaches to the name resolution problem after 1.0. +# Miscellaneous remarks + +`macro` is no longer a macro invocation, but we don't attempt to parse its +contents any more than we currently parse `macro_rules!`. The item AST type +changes to + +```rust +pub enum Item_ { + // ... + + /// A macro definition. + ItemDefineMacro(Vec), + + /// A macro invocation. + ItemUseMacro(Mac), // was called ItemMac +} +``` + +The current `macro_rules!`, which parses as an `ItemMac`. + +Leaving the body as an uninterpreted token tree also provides flexibility to +make backwards-compatible changes. One can imagine + +```rust +#[procedural] macro match_token { + fn expand(cx: &mut ExtCtxt, span: Span, toks: &[TokenTree]) + -> Box { + // ... + } +} +``` + +or + +```rust +macro atom { + ($name:tt) => fn expand(...) { + // ... + } +} +``` + +though working out the details is far outside the scope of this RFC. + +We are free to change the AST and parsing after 1.0 as long as the old syntax +still works. So we could have a separate enum variant for procedural macros +parsed as function decls. + +In a future where macros are scoped the same way as other items, + +```rust +use macro std::vec; +``` + +would become a deprecated synonym for + +```rust +use std::vec!; +``` + +Maintaining this synonym does not seem like a large burden. + +We can add a lint to warn about cases where an `extern` macro has paths that +are not absolute-with-crate or `$crate`-relative. This will have some +(hopefully rare) false positives, and is not fully fleshed out yet. + +# Implementation and transition + +I will coordinate implementation of this RFC, and I expect to write most of the +code myself. + +Some of the syntax cleanups could be deferred until after 1.0. However the +semantic changes are enough that many users will re-examine and perhaps edit a +large fraction of their macro definitions. My thinking is that the cleaned-up +syntax should be available when this happens rather than requiring a separate +pass a few months later. Also I would really like the first Rust release to +put its best foot forward with macros, not just in terms of semantics but +with a polished and pleasant user experience. + +To ease the transition, we can keep the old `macro_rules!` syntax as a +deprecated synonym, to be removed before 1.0. + # Drawbacks This is big churn on a major feature, not long before 1.0. @@ -398,6 +443,12 @@ Should we forbid `$crate` in non-`extern` macros? It seems useless, however I think we should allow it anyway, to encourage the habit of writing `$crate::` for any references to the local crate. +Should we allow `pub use macro`, which would escape the enclosing block/module +the way `pub macro` does? + +Should we require that `extern use macro` can only appear in the crate root? +It doesn't make a lot of sense to put it elsewhere. + # Acknowledgements This proposal is edited by Keegan McAllister. It has been refined through many From 7a25233e66415e7e5f8c0f92a7d740dcfe94cb49 Mon Sep 17 00:00:00 2001 From: Keegan McAllister Date: Sun, 16 Nov 2014 21:47:54 -0800 Subject: [PATCH 17/27] Remove leftover partial sentence --- text/0000-macro-reform.md | 2 -- 1 file changed, 2 deletions(-) diff --git a/text/0000-macro-reform.md b/text/0000-macro-reform.md index 8d2840545ad..01db476d5e0 100644 --- a/text/0000-macro-reform.md +++ b/text/0000-macro-reform.md @@ -337,8 +337,6 @@ pub enum Item_ { } ``` -The current `macro_rules!`, which parses as an `ItemMac`. - Leaving the body as an uninterpreted token tree also provides flexibility to make backwards-compatible changes. One can imagine From 0471d6179d425e54f2f57ec4f0c57adb85439a45 Mon Sep 17 00:00:00 2001 From: Keegan McAllister Date: Wed, 19 Nov 2014 15:11:16 -0800 Subject: [PATCH 18/27] Stick with macro_rules! syntax for definitions --- text/0000-macro-reform.md | 111 ++++++++++---------------------------- 1 file changed, 27 insertions(+), 84 deletions(-) diff --git a/text/0000-macro-reform.md b/text/0000-macro-reform.md index 01db476d5e0..109830acce6 100644 --- a/text/0000-macro-reform.md +++ b/text/0000-macro-reform.md @@ -22,8 +22,6 @@ edges](https://github.com/rust-lang/rfcs/issues/440). A few of the big ones: - There's no namespacing at all - You can't control which macros are imported from a crate - You need the feature-gated `#[phase(plugin)]` to import macros -- It's confusing that macro definition is itself a macro invocation, with a side effect - on the syntax environment These issues in particular are things we have a chance of addressing for 1.0. This RFC contains plans to do so. @@ -74,10 +72,10 @@ definition. ```rust /// Not to be imported directly. -extern macro lint_initializer { ... } +extern macro_rules! lint_initializer { ... } /// Declare a lint. -extern macro declare_lint { +extern macro_rules! declare_lint { // See below for $crate use macro $crate::lint_initializer; @@ -109,45 +107,34 @@ rebase. We also clean up macro syntax in a way that complements the semantic changes above. -## `macro_rules!` becomes `macro` +## Macro definition syntax -The new macro definition syntax uses the `macro` keyword introduced above: - -```rust -// first example from the Macros Guide -macro early_return { - ($inp:expr $sp:ident) => ( - match $inp { - $sp(x) => { return x; } - _ => {} - } - ) -} -``` - -This is an ordinary item, like `fn` or `struct`, not a macro invocation. It -defines a new macro in the syntax environment of the enclosing block or module. +`macro_rules!` already allows `{ }` for the macro body, but the convention is +`( )` for some reason. In accepting this RFC we would change to a `{ }` +convention for consistency with the rest of the language. The new macro can be used immediately. There is no way to `use macro` a macro defined in the same crate, and no need to do so. -If qualified by `pub`, the macro escapes the syntax environment for the -enclosing block/module and becomes available throughout the rest of the crate -(according to depth-first search). This is like putting `#[macro_escape]` on -the module and all its ancestors, but applies *only* to the macro with `pub`. +A macro with a `pub` qualifier, i.e. -`macro_rules!` already allows `{ }` for the macro body, but the convention is -`( )` for some reason. In accepting this RFC we would change to a `{ }` -convention for consistency with the rest of the language. +```rust +pub macro_rules! foo { ... } +``` + +escapes the syntax environment for the enclosing block/module and becomes +available throughout the rest of the crate (according to depth-first search). +This is like putting `#[macro_escape]` on the module and all its ancestors, but +applies *only* to the macro with `pub`. ## Macro export and re-export -A `macro` item qualified by `extern` becomes available to other crates. That -is, it can be the target of `use macro`. Or put another way, `extern macro` -works the way `#[macro_export] macro_rules!` does today. Adding `extern` has -no effect on the syntax environment for the current crate. +A macro definition qualified by `extern` becomes available to other crates. +That is, it can be the target of `use macro`. Or put another way, +`extern macro_rules!` works the way `#[macro_export] macro_rules!` does today. +Adding `extern` has no effect on the syntax environment for the current crate. -`pub` and `extern` may be used together on the same `macro` definition, since +`pub` and `extern` may be used together on the same macro definition, since their effects are independent. We can also re-export macros that were imported from another crate. This is @@ -156,7 +143,7 @@ accomplished with `extern use macro`. For example, libcollections defines a `vec!` macro, which would now look like: ```rust -extern macro vec { +extern macro_rules! vec { ($($e:expr),*) => ({ let mut _temp = $crate::vec::Vec::new(); $(_temp.push($e);)* @@ -220,9 +207,9 @@ Item macro sugar is an integral part of macro reform because it's needed to make `pub` and `extern` work with macro-defining macros. For example ```rust -macro mega_macro { +macro_rules! mega_macro { ($($qual:ident)* : $name:ident $body:tt) => ( - $($qual)* macro $name { + $($qual)* macro_rules! $name { // ... } ) @@ -237,7 +224,7 @@ pub extern mega_macro! foo { } ``` -Here `mega_macro!` takes the place of the built-in `macro` keyword. +Here `mega_macro!` takes the place of the built-in `macro_rules!`. The applications go beyond macro-defining macros. For example, [this macro in rustc](https://github.com/rust-lang/rust/blob/221fc1e3cdcc208e1bb7debcc2de27d47c847747/src/librustc/lint/mod.rs#L83-L95) @@ -321,50 +308,6 @@ name resolution problem after 1.0. # Miscellaneous remarks -`macro` is no longer a macro invocation, but we don't attempt to parse its -contents any more than we currently parse `macro_rules!`. The item AST type -changes to - -```rust -pub enum Item_ { - // ... - - /// A macro definition. - ItemDefineMacro(Vec), - - /// A macro invocation. - ItemUseMacro(Mac), // was called ItemMac -} -``` - -Leaving the body as an uninterpreted token tree also provides flexibility to -make backwards-compatible changes. One can imagine - -```rust -#[procedural] macro match_token { - fn expand(cx: &mut ExtCtxt, span: Span, toks: &[TokenTree]) - -> Box { - // ... - } -} -``` - -or - -```rust -macro atom { - ($name:tt) => fn expand(...) { - // ... - } -} -``` - -though working out the details is far outside the scope of this RFC. - -We are free to change the AST and parsing after 1.0 as long as the old syntax -still works. So we could have a separate enum variant for procedural macros -parsed as function decls. - In a future where macros are scoped the same way as other items, ```rust @@ -396,8 +339,8 @@ pass a few months later. Also I would really like the first Rust release to put its best foot forward with macros, not just in terms of semantics but with a polished and pleasant user experience. -To ease the transition, we can keep the old `macro_rules!` syntax as a -deprecated synonym, to be removed before 1.0. +To ease the transition, we can keep the old syntax as a deprecated synonym, to +be removed before 1.0. # Drawbacks @@ -442,7 +385,7 @@ think we should allow it anyway, to encourage the habit of writing `$crate::` for any references to the local crate. Should we allow `pub use macro`, which would escape the enclosing block/module -the way `pub macro` does? +the way `pub macro_rules!` does? Should we require that `extern use macro` can only appear in the crate root? It doesn't make a lot of sense to put it elsewhere. From ee9e2e8f657a7d0788883b55df63d487899c336a Mon Sep 17 00:00:00 2001 From: Keegan McAllister Date: Wed, 19 Nov 2014 15:11:48 -0800 Subject: [PATCH 19/27] Remove item macro sugar This can wait for a future RFC. --- text/0000-macro-reform.md | 82 --------------------------------------- 1 file changed, 82 deletions(-) diff --git a/text/0000-macro-reform.md b/text/0000-macro-reform.md index 109830acce6..2ded9404e1c 100644 --- a/text/0000-macro-reform.md +++ b/text/0000-macro-reform.md @@ -169,88 +169,6 @@ Because macros are exported in crate metadata as strings, macro re-export "just works" as soon as `$crate` is available. It's implemented as part of the `$crate` branch mentioned above. -## Item macro sugar - -In Rust's syntax, an item defines one name in the current module, and can have -"adjective" qualifiers such as `pub`, `extern`, etc. We extend macro -invocation in item position to reflect this form. The new invocation syntax is - -```rust - foo! { - -} -``` - -where `` is a sequence of zero or more keywords from the set `pub` -`priv` `extern` `unsafe` `const` `static` `box` `ref` `mut`. This list is -pretty arbitrary but can be expanded later. For now it only includes keywords -that have existing meaning(s) in Rust, which should keep things somewhat under -control. - -This form of item macro is a bit like defining your own keyword that can take -the place of `fn`, `struct`, etc. - -We keep the existing syntax for invoking macros in item position, and desugar -the above syntax to it, as - -```rust -foo!( : { }) -``` - -This extends the existing `IdentTT` macro form and makes it available to -non-procedural macros, while reducing special cases within the compiler. - -An item macro invocation in either the sugared or unsugared form may expand to -zero or more items. - -Item macro sugar is an integral part of macro reform because it's needed to -make `pub` and `extern` work with macro-defining macros. For example - -```rust -macro_rules! mega_macro { - ($($qual:ident)* : $name:ident $body:tt) => ( - $($qual)* macro_rules! $name { - // ... - } - ) -} -``` - -can be invoked as - -```rust -pub extern mega_macro! foo { - (bar) => (3u) -} -``` - -Here `mega_macro!` takes the place of the built-in `macro_rules!`. - -The applications go beyond macro-defining macros. For example, [this macro in -rustc](https://github.com/rust-lang/rust/blob/221fc1e3cdcc208e1bb7debcc2de27d47c847747/src/librustc/lint/mod.rs#L83-L95) -could change to support invocations like - -```rust -lint! UNUSED_ATTRIBUTES { - Warn, "detects attributes that were not used by the compiler" -} - -pub lint! PATH_STATEMENTS { - Warn, "path statements with no effect" -} -``` - -which is considerably nicer than the current syntax. -[`lazy-static.rs`](https://github.com/Kimundi/lazy-static.rs) could support -syntax like - -```rust -pub lazy_static! LOG { - : Mutex> - = Mutex::new(Vec::with_capacity(50_000)) -} -``` - ## `#[plugin]` attribute Since macros no longer automatically pollute the syntax environment, we can From 0c108713ea69e57d212a821f19678b0f73dccef1 Mon Sep 17 00:00:00 2001 From: Keegan McAllister Date: Wed, 19 Nov 2014 15:32:16 -0800 Subject: [PATCH 20/27] Remove section about procedural macro order This can wait. --- text/0000-macro-reform.md | 21 --------------------- 1 file changed, 21 deletions(-) diff --git a/text/0000-macro-reform.md b/text/0000-macro-reform.md index 2ded9404e1c..a2148e05e82 100644 --- a/text/0000-macro-reform.md +++ b/text/0000-macro-reform.md @@ -203,27 +203,6 @@ This facilitates plugin configuration. The alternative in many cases is to use interacting side effects between procedural macros, which are harder to reason about. -## Unspecify order of procedural macro side effects - -This is basically an internal-only change, but I mention it in this RFC (for -now) because it was discussed alongside the rest of this. - -We clarify that the ordering of expansion, hence side effects, for separate -(i.e. non-nested) procedural macro invocations is unspecified. This does not -affect the stable language, because procedural macros are not part of it, and -expansion of pattern-based macros cannot have side effects. - -Interacting side effects between procedural macros is messy in general. It's -much better for a macro to approximate a pure function of its input, plus an -"environment" that does not change during macro expansion. I claim that this -is usually possible (see -[discussion](https://github.com/rust-lang/rfcs/pull/453#issuecomment-62813856)). - -The main reason to consider this now in an RFC is to make sure that built-in -procedural macros exposed to stable code can comply. Reserving the right to -change expansion order allows us to pursue more sophisticated approaches to the -name resolution problem after 1.0. - # Miscellaneous remarks In a future where macros are scoped the same way as other items, From ee22f7f5d95dbdef0de3ffe9dd9301ded31aaa54 Mon Sep 17 00:00:00 2001 From: Keegan McAllister Date: Wed, 19 Nov 2014 15:38:41 -0800 Subject: [PATCH 21/27] Clarify the "crate ident" concept --- text/0000-macro-reform.md | 51 ++++++++++++++++++++++++--------------- 1 file changed, 32 insertions(+), 19 deletions(-) diff --git a/text/0000-macro-reform.md b/text/0000-macro-reform.md index a2148e05e82..26c3c5875bf 100644 --- a/text/0000-macro-reform.md +++ b/text/0000-macro-reform.md @@ -31,6 +31,28 @@ This RFC contains plans to do so. These are the substantial changes to the macro system. The examples also use the improved syntax, described later. +## `$crate` + +The first change is to disallow importing macros from an `extern crate` that is +not at the crate root. In that case, if + +```rust +extern crate "bar" as foo; +``` + +imports macros, then it's also introducing ordinary paths of the form +`::foo::...`. We call `foo` the *crate ident* of the `extern crate`. + +We introduce a special macro metavar `$crate` which expands to `::foo` when a +macro was imported through crate ident `foo`, and to nothing when it was +defined in the crate where it is being expanded. `$crate::bar::baz` will be an +absolute path either way. + +This feature eliminates the need for the "curious inner-module" and also +enables macro re-export (see below). It is [implemented and +tested](https://github.com/kmcallister/rust/commits/macro-reexport) but needs a +rebase. + ## Crate scope for macros In this document, the "syntax environment" refers to the set of syntax @@ -41,10 +63,10 @@ However, the exclamation point is really part of the invocation syntax, not the name, and some syntax extensions are invoked with no exclamation point, for example item decorators like `deriving`. -The first proposed change is that macros imported from another crate will not -automatically end up in the syntax environment, as they do today. Instead, you -can bring an imported macro into the syntax environment with a new view item, -`use macro`: +Imported macros will not automatically end up in the syntax environment. +Instead, you can bring an imported macro into the syntax environment by +providing the crate ident and a macro name (or wildcard) in a `use macro` view +item: ```rust use macro std::vec; @@ -52,13 +74,16 @@ use macro std::panic as fail; use macro core::*; ``` +These paths must have exactly two components: the crate ident, and a macro name +or `*`. + The syntax environment still consists of unqualified names. There's no way to invoke a macro through a qualified name. This obviates the need to change the parsing of expressions, patterns, etc. -The `macro` keyword is important because it signals that this is something -different from the usual name resolution. `use macro` is a memorable and -searchable name for the feature. +`macro` is a new keyword. This is an important part of the proposal, because +it signals that we're deviating from usual name resolution. `use macro` is a +memorable and searchable name for the feature. The `use macro` view item only affects the syntax environment of the block or module where it appears, and only from the point of appearance onward. Unlike @@ -76,7 +101,6 @@ extern macro_rules! lint_initializer { ... } /// Declare a lint. extern macro_rules! declare_lint { - // See below for $crate use macro $crate::lint_initializer; ($name:ident, $level:ident, $desc:expr) => ( @@ -92,17 +116,6 @@ the result of invoking `declare_lint!`. Procedural macros need their own way to manipulate the syntax environment, but that's an unstable internal API, so it's outside the scope of this RFC. -## `$crate` - -We add a special metavar `$crate` which expands to `::foo` when a macro was -imported from crate `foo`, and to nothing when it was defined in-crate. -`$crate::bar::baz` will be an absolute path either way. - -This feature eliminates the need for the "curious inner-module" and also -enables macro re-export (see below). It is [implemented and -tested](https://github.com/kmcallister/rust/commits/macro-reexport) but needs a -rebase. - # New syntax We also clean up macro syntax in a way that complements the semantic changes above. From c6cdf8c2aef376dd1e06202d51640a352259e3aa Mon Sep 17 00:00:00 2001 From: Keegan McAllister Date: Wed, 19 Nov 2014 15:39:30 -0800 Subject: [PATCH 22/27] Reorganize #[plugin] section --- text/0000-macro-reform.md | 24 ++++++++---------------- 1 file changed, 8 insertions(+), 16 deletions(-) diff --git a/text/0000-macro-reform.md b/text/0000-macro-reform.md index 26c3c5875bf..a11bdb5bb2c 100644 --- a/text/0000-macro-reform.md +++ b/text/0000-macro-reform.md @@ -194,27 +194,19 @@ only controls whether to search for and run a plugin registrar function. The plugin itself will decide whether it's to be linked at runtime, by calling a `Registry` method. -# Other improvements - -These are somewhat related to the above, but could be spun off into separate -RFCs. - -## Arguments to `#[plugin]` - -`#[plugin]` will take an optional "arguments list" of the form +`#[plugin]` can optionally take any [meta +items](http://doc.rust-lang.org/syntax/ast/enum.MetaItem_.html) as "arguments", +e.g. ```rust -#[plugin(foo="bar", ... any metas ...)] +#[plugin(foo, bar=3, baz(quux))] extern crate myplugin; ``` -rustc itself will not interpret these attribute [meta -items](http://doc.rust-lang.org/syntax/ast/enum.MetaItem_.html), but will make -them available to the plugin through a `Registry` method. - -This facilitates plugin configuration. The alternative in many cases is to use -interacting side effects between procedural macros, which are harder to reason -about. +rustc itself will not interpret these arguments, but will make them available +to the plugin through a `Registry` method. This facilitates plugin +configuration. The alternative in many cases is to use interacting side +effects between procedural macros, which are harder to reason about. # Miscellaneous remarks From 45ff6ecd2228d11abc62832400450bf113dd303b Mon Sep 17 00:00:00 2001 From: Keegan McAllister Date: Wed, 19 Nov 2014 15:42:34 -0800 Subject: [PATCH 23/27] More acknowledgements --- text/0000-macro-reform.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/text/0000-macro-reform.md b/text/0000-macro-reform.md index a11bdb5bb2c..cabec111411 100644 --- a/text/0000-macro-reform.md +++ b/text/0000-macro-reform.md @@ -299,7 +299,7 @@ engaging discussions with: * Brian Anderson, Shachaf Ben-Kiki, Lars Bergstrom, Nick Cameron, John Clements, Alex Crichton, Cathy Douglass, Steven Fackler, Manish Goregaokar, Dave Herman, Steve Klabnik, Felix S. Klock II, Niko Matsakis, Matthew McPherrin, Paul Stansifer, Sam Tobin-Hochstadt, Aaron Turon, Huon Wilson, Brendan Zabarauskas, Cameron Zwarich * *GitHub*: `@bill-myers` `@blaenk` `@comex` `@glaebhoerl` `@Kimundi` `@mitchmindtree` `@mitsuhiko` `@P1Start` `@petrochenkov` `@skinner` -* *Reddit*: `ippa` `Mystor` `Quxxy` `rime-frost` `Sinistersnare` `tejp` +* *Reddit*: `gnusouth` `ippa` `!kibwen` `Mystor` `Quxxy` `rime-frost` `Sinistersnare` `tejp` `UtherII` `yigal100` * *IRC*: `bstrie` `ChrisMorgan` `cmr` `Earnestly` `eddyb` `tiffany` My apologies if I've forgotten you, used an un-preferred name, or accidentally From 3e5861db89990c35e0944146903003a6b3d7efdc Mon Sep 17 00:00:00 2001 From: Keegan McAllister Date: Tue, 25 Nov 2014 17:53:09 -0800 Subject: [PATCH 24/27] New revision --- text/0000-macro-reform.md | 158 +++++++++++++++----------------------- 1 file changed, 63 insertions(+), 95 deletions(-) diff --git a/text/0000-macro-reform.md b/text/0000-macro-reform.md index cabec111411..39f90abcc80 100644 --- a/text/0000-macro-reform.md +++ b/text/0000-macro-reform.md @@ -53,7 +53,11 @@ enables macro re-export (see below). It is [implemented and tested](https://github.com/kmcallister/rust/commits/macro-reexport) but needs a rebase. -## Crate scope for macros +We can add a lint to warn about cases where an exported macro has paths that +are not absolute-with-crate or `$crate`-relative. This will have some +(hopefully rare) false positives. + +## Macro scope In this document, the "syntax environment" refers to the set of syntax extensions that can be invoked at a given position in the crate. The names in @@ -63,46 +67,38 @@ However, the exclamation point is really part of the invocation syntax, not the name, and some syntax extensions are invoked with no exclamation point, for example item decorators like `deriving`. -Imported macros will not automatically end up in the syntax environment. -Instead, you can bring an imported macro into the syntax environment by -providing the crate ident and a macro name (or wildcard) in a `use macro` view -item: +We introduce an attribute `use_macros` to specify which macros from an external +crate should be imported to the syntax environment: ```rust -use macro std::vec; -use macro std::panic as fail; -use macro core::*; -``` - -These paths must have exactly two components: the crate ident, and a macro name -or `*`. +#[use_macros(vec, panic="fail")] +extern crate std; -The syntax environment still consists of unqualified names. There's no way to -invoke a macro through a qualified name. This obviates the need to change the -parsing of expressions, patterns, etc. - -`macro` is a new keyword. This is an important part of the proposal, because -it signals that we're deviating from usual name resolution. `use macro` is a -memorable and searchable name for the feature. +#[use_macros(*)] +extern crate core; +``` -The `use macro` view item only affects the syntax environment of the block or -module where it appears, and only from the point of appearance onward. Unlike -a normal `use` item, this includes child modules (in the same file or others). +Macros imported this way can be used anywhere in the module after the +`extern crate` item, including in child modules. Since a macro-importing +`extern crate` must appear at the crate root, and view items come before +other items, this effectively means imported macros will be visible for +the entire crate. -Many macros expand using other "private macros" as an implementation detail. +Many macros expand using other "helper macros" as an implementation detail. For example, librustc's `declare_lint!` uses `lint_initializer!`. The client should not know about this macro, although it still needs to be exported for -cross-crate use. For this reason we allow `use macro` within a macro +cross-crate use. For this reason we allow `#[use_macros]` on a macro definition. ```rust /// Not to be imported directly. -extern macro_rules! lint_initializer { ... } +#[export] +macro_rules! lint_initializer { ... } /// Declare a lint. -extern macro_rules! declare_lint { - use macro $crate::lint_initializer; - +#[export] +#[use_macros(lint_initializer)] +macro_rules! declare_lint { ($name:ident, $level:ident, $desc:expr) => ( static $name: &'static $crate::lint::Lint = &lint_initializer!($name, $level, $desc); @@ -110,8 +106,9 @@ extern macro_rules! declare_lint { } ``` -The macro `lint_initializer!` will be visible only during further expansion of -the result of invoking `declare_lint!`. +The macro `lint_initializer!`, imported from the same crate as `declare_lint!`, +will be visible only during further expansion of the result of invoking +`declare_lint!`. Procedural macros need their own way to manipulate the syntax environment, but that's an unstable internal API, so it's outside the scope of this RFC. @@ -120,43 +117,40 @@ that's an unstable internal API, so it's outside the scope of this RFC. We also clean up macro syntax in a way that complements the semantic changes above. -## Macro definition syntax +## `#[use_macros(...)] mod` -`macro_rules!` already allows `{ }` for the macro body, but the convention is -`( )` for some reason. In accepting this RFC we would change to a `{ }` -convention for consistency with the rest of the language. - -The new macro can be used immediately. There is no way to `use macro` a macro -defined in the same crate, and no need to do so. - -A macro with a `pub` qualifier, i.e. +The `use_macros` attribute can be applied to a `mod` item as well. The +specified macros will "escape" the module and become visible throughout the +rest of the enclosing module, including any child modules. A crate might start +with ```rust -pub macro_rules! foo { ... } +#[use_macros(*)] +mod macros; ``` -escapes the syntax environment for the enclosing block/module and becomes -available throughout the rest of the crate (according to depth-first search). -This is like putting `#[macro_escape]` on the module and all its ancestors, but -applies *only* to the macro with `pub`. +to define some macros for use by the whole crate, without putting those +definitions in `lib.rs`. -## Macro export and re-export - -A macro definition qualified by `extern` becomes available to other crates. -That is, it can be the target of `use macro`. Or put another way, -`extern macro_rules!` works the way `#[macro_export] macro_rules!` does today. -Adding `extern` has no effect on the syntax environment for the current crate. +Note that `#[use_macros(*)]` is equivalent to the current `#[macro_escape]`. +However, the new convention is to use an outer attribute, in the file whose +syntax environment is affected, rather than an inner attribute in the file +defining the macros. -`pub` and `extern` may be used together on the same macro definition, since -their effects are independent. +## Macro export and re-export -We can also re-export macros that were imported from another crate. This is -accomplished with `extern use macro`. +A macro definition qualified by `#[export]` becomes available to other crates. +That is, it can be the target of `#[use_macros]`. Or put another way, +`#[export] macro_rules!` works the way `#[macro_export] macro_rules!` does +today. Adding `#[export]` has no effect on the syntax environment for the +current crate. -For example, libcollections defines a `vec!` macro, which would now look like: +We can also re-export macros that were imported from another crate. For +example, libcollections defines a `vec!` macro, which would now look like: ```rust -extern macro_rules! vec { +#[export] +macro_rules! vec { ($($e:expr),*) => ({ let mut _temp = $crate::vec::Vec::new(); $(_temp.push($e);)* @@ -169,7 +163,8 @@ Currently, libstd duplicates this macro in its own `macros.rs`. Now it could do ```rust -extern use macro collections::vec; +#[reexport_macros(vec)] +extern crate collections; ``` as long as the module `std::vec` is interface-compatible with @@ -184,11 +179,6 @@ works" as soon as `$crate` is available. It's implemented as part of the ## `#[plugin]` attribute -Since macros no longer automatically pollute the syntax environment, we can -load them from every `extern crate`. (Probably we should exclude -`extern crate`s that aren't at the crate root, because there's no way `$crate` -paths will be correct.) - `#[phase(plugin)]` becomes simply `#[plugin]` and is still feature-gated. It only controls whether to search for and run a plugin registrar function. The plugin itself will decide whether it's to be linked at runtime, by calling a @@ -208,39 +198,23 @@ to the plugin through a `Registry` method. This facilitates plugin configuration. The alternative in many cases is to use interacting side effects between procedural macros, which are harder to reason about. -# Miscellaneous remarks - -In a future where macros are scoped the same way as other items, - -```rust -use macro std::vec; -``` - -would become a deprecated synonym for +## Syntax convention -```rust -use std::vec!; -``` +`macro_rules!` already allows `{ }` for the macro body, but the convention is +`( )` for some reason. In accepting this RFC we would change to a `{ }` +convention for consistency with the rest of the language. -Maintaining this synonym does not seem like a large burden. +## Reserve `macro` as a keyword -We can add a lint to warn about cases where an `extern` macro has paths that -are not absolute-with-crate or `$crate`-relative. This will have some -(hopefully rare) false positives, and is not fully fleshed out yet. +A lot of the syntax alternatives discussed for this RFC involved a `macro` +keyword. The consensus is that macros are too unfinished to merit using the +keyword now. However, we should reserve it for a future macro system. # Implementation and transition I will coordinate implementation of this RFC, and I expect to write most of the code myself. -Some of the syntax cleanups could be deferred until after 1.0. However the -semantic changes are enough that many users will re-examine and perhaps edit a -large fraction of their macro definitions. My thinking is that the cleaned-up -syntax should be available when this happens rather than requiring a separate -pass a few months later. Also I would really like the first Rust release to -put its best foot forward with macros, not just in terms of semantics but -with a polished and pleasant user experience. - To ease the transition, we can keep the old syntax as a deprecated synonym, to be removed before 1.0. @@ -282,15 +256,9 @@ reform. Two ways this could work out: # Unresolved questions -Should we forbid `$crate` in non-`extern` macros? It seems useless, however I -think we should allow it anyway, to encourage the habit of writing `$crate::` -for any references to the local crate. - -Should we allow `pub use macro`, which would escape the enclosing block/module -the way `pub macro_rules!` does? - -Should we require that `extern use macro` can only appear in the crate root? -It doesn't make a lot of sense to put it elsewhere. +Should we forbid `$crate` in non-`#[export]`ed macros? It seems useless, +however I think we should allow it anyway, to encourage the habit of writing +`$crate::` for any references to the local crate. # Acknowledgements From a503fb4c5abb08add7186b5c59a08b6a5e20acf7 Mon Sep 17 00:00:00 2001 From: Keegan McAllister Date: Thu, 4 Dec 2014 16:48:07 -0800 Subject: [PATCH 25/27] Changes from the meeting --- text/0000-macro-reform.md | 29 ++++++++++++++++++++++------- 1 file changed, 22 insertions(+), 7 deletions(-) diff --git a/text/0000-macro-reform.md b/text/0000-macro-reform.md index 39f90abcc80..552eeb1e627 100644 --- a/text/0000-macro-reform.md +++ b/text/0000-macro-reform.md @@ -74,16 +74,26 @@ crate should be imported to the syntax environment: #[use_macros(vec, panic="fail")] extern crate std; -#[use_macros(*)] +#[use_macros] extern crate core; ``` +The list of macros to import is optional. Omitting the list imports all macros, +similar to a glob `use`. (This is also the mechanism by which `std` will +inject its macros into every non-`no_std` crate.) + +Importing with rename is an optional part of this proposal that will be +implemented for 1.0 only if time permits. + Macros imported this way can be used anywhere in the module after the `extern crate` item, including in child modules. Since a macro-importing `extern crate` must appear at the crate root, and view items come before other items, this effectively means imported macros will be visible for the entire crate. +Any name collision between macros, whether imported or defined in-crate, is a +hard error. + Many macros expand using other "helper macros" as an implementation detail. For example, librustc's `declare_lint!` uses `lint_initializer!`. The client should not know about this macro, although it still needs to be exported for @@ -110,6 +120,11 @@ The macro `lint_initializer!`, imported from the same crate as `declare_lint!`, will be visible only during further expansion of the result of invoking `declare_lint!`. +`use_macros` on `macro_rules` is an optional part of this proposal that will be +implemented for 1.0 only if time permits. Without it, libraries that use +helper macros will need to list them in documentation so that users can import +them. + Procedural macros need their own way to manipulate the syntax environment, but that's an unstable internal API, so it's outside the scope of this RFC. @@ -125,17 +140,17 @@ rest of the enclosing module, including any child modules. A crate might start with ```rust -#[use_macros(*)] +#[use_macros] mod macros; ``` to define some macros for use by the whole crate, without putting those definitions in `lib.rs`. -Note that `#[use_macros(*)]` is equivalent to the current `#[macro_escape]`. -However, the new convention is to use an outer attribute, in the file whose -syntax environment is affected, rather than an inner attribute in the file -defining the macros. +Note that `#[use_macros]` (without a list of names) is equivalent to the +current `#[macro_escape]`. However, the new convention is to use an outer +attribute, in the file whose syntax environment is affected, rather than an +inner attribute in the file defining the macros. ## Macro export and re-export @@ -265,7 +280,7 @@ however I think we should allow it anyway, to encourage the habit of writing This proposal is edited by Keegan McAllister. It has been refined through many engaging discussions with: -* Brian Anderson, Shachaf Ben-Kiki, Lars Bergstrom, Nick Cameron, John Clements, Alex Crichton, Cathy Douglass, Steven Fackler, Manish Goregaokar, Dave Herman, Steve Klabnik, Felix S. Klock II, Niko Matsakis, Matthew McPherrin, Paul Stansifer, Sam Tobin-Hochstadt, Aaron Turon, Huon Wilson, Brendan Zabarauskas, Cameron Zwarich +* Brian Anderson, Shachaf Ben-Kiki, Lars Bergstrom, Nick Cameron, John Clements, Alex Crichton, Cathy Douglass, Steven Fackler, Manish Goregaokar, Dave Herman, Steve Klabnik, Felix S. Klock II, Niko Matsakis, Matthew McPherrin, Paul Stansifer, Sam Tobin-Hochstadt, Erick Tryzelaar, Aaron Turon, Huon Wilson, Brendan Zabarauskas, Cameron Zwarich * *GitHub*: `@bill-myers` `@blaenk` `@comex` `@glaebhoerl` `@Kimundi` `@mitchmindtree` `@mitsuhiko` `@P1Start` `@petrochenkov` `@skinner` * *Reddit*: `gnusouth` `ippa` `!kibwen` `Mystor` `Quxxy` `rime-frost` `Sinistersnare` `tejp` `UtherII` `yigal100` * *IRC*: `bstrie` `ChrisMorgan` `cmr` `Earnestly` `eddyb` `tiffany` From ebc0a8527dd80e77b22ffad5a6c09e00bb59e569 Mon Sep 17 00:00:00 2001 From: Keegan McAllister Date: Thu, 4 Dec 2014 17:20:23 -0800 Subject: [PATCH 26/27] Rename attributes per meeting --- text/0000-macro-reform.md | 47 +++++++++++++++++++++------------------ 1 file changed, 25 insertions(+), 22 deletions(-) diff --git a/text/0000-macro-reform.md b/text/0000-macro-reform.md index 552eeb1e627..e6fdf17b6a2 100644 --- a/text/0000-macro-reform.md +++ b/text/0000-macro-reform.md @@ -67,14 +67,14 @@ However, the exclamation point is really part of the invocation syntax, not the name, and some syntax extensions are invoked with no exclamation point, for example item decorators like `deriving`. -We introduce an attribute `use_macros` to specify which macros from an external +We introduce an attribute `macro_use` to specify which macros from an external crate should be imported to the syntax environment: ```rust -#[use_macros(vec, panic="fail")] +#[macro_use(vec, panic="fail")] extern crate std; -#[use_macros] +#[macro_use] extern crate core; ``` @@ -97,17 +97,17 @@ hard error. Many macros expand using other "helper macros" as an implementation detail. For example, librustc's `declare_lint!` uses `lint_initializer!`. The client should not know about this macro, although it still needs to be exported for -cross-crate use. For this reason we allow `#[use_macros]` on a macro +cross-crate use. For this reason we allow `#[macro_use]` on a macro definition. ```rust /// Not to be imported directly. -#[export] +#[macro_export] macro_rules! lint_initializer { ... } /// Declare a lint. -#[export] -#[use_macros(lint_initializer)] +#[macro_export] +#[macro_use(lint_initializer)] macro_rules! declare_lint { ($name:ident, $level:ident, $desc:expr) => ( static $name: &'static $crate::lint::Lint @@ -120,7 +120,7 @@ The macro `lint_initializer!`, imported from the same crate as `declare_lint!`, will be visible only during further expansion of the result of invoking `declare_lint!`. -`use_macros` on `macro_rules` is an optional part of this proposal that will be +`macro_use` on `macro_rules` is an optional part of this proposal that will be implemented for 1.0 only if time permits. Without it, libraries that use helper macros will need to list them in documentation so that users can import them. @@ -132,39 +132,42 @@ that's an unstable internal API, so it's outside the scope of this RFC. We also clean up macro syntax in a way that complements the semantic changes above. -## `#[use_macros(...)] mod` +## `#[macro_use(...)] mod` -The `use_macros` attribute can be applied to a `mod` item as well. The +The `macro_use` attribute can be applied to a `mod` item as well. The specified macros will "escape" the module and become visible throughout the rest of the enclosing module, including any child modules. A crate might start with ```rust -#[use_macros] +#[macro_use] mod macros; ``` to define some macros for use by the whole crate, without putting those definitions in `lib.rs`. -Note that `#[use_macros]` (without a list of names) is equivalent to the +Note that `#[macro_use]` (without a list of names) is equivalent to the current `#[macro_escape]`. However, the new convention is to use an outer attribute, in the file whose syntax environment is affected, rather than an inner attribute in the file defining the macros. ## Macro export and re-export -A macro definition qualified by `#[export]` becomes available to other crates. -That is, it can be the target of `#[use_macros]`. Or put another way, -`#[export] macro_rules!` works the way `#[macro_export] macro_rules!` does -today. Adding `#[export]` has no effect on the syntax environment for the -current crate. +Currently in Rust, a macro definition qualified by `#[macro_export]` becomes +available to other crates. We keep this behavior in the new system. A macro +qualified by `#[macro_export]` can be the target of `#[macro_use(...)]`, and +will be imported automatically when `#[macro_use]` is given with no list of +names. + +`#[macro_export]` has no effect on the syntax environment for the current +crate. We can also re-export macros that were imported from another crate. For example, libcollections defines a `vec!` macro, which would now look like: ```rust -#[export] +#[macro_export] macro_rules! vec { ($($e:expr),*) => ({ let mut _temp = $crate::vec::Vec::new(); @@ -178,7 +181,7 @@ Currently, libstd duplicates this macro in its own `macros.rs`. Now it could do ```rust -#[reexport_macros(vec)] +#[macro_reexport(vec)] extern crate collections; ``` @@ -271,9 +274,9 @@ reform. Two ways this could work out: # Unresolved questions -Should we forbid `$crate` in non-`#[export]`ed macros? It seems useless, -however I think we should allow it anyway, to encourage the habit of writing -`$crate::` for any references to the local crate. +Should we forbid `$crate` in non-exported macros? It seems useless, however I +think we should allow it anyway, to encourage the habit of writing `$crate::` +for any references to the local crate. # Acknowledgements From 2f6bdc8b80e2b49bda8382649e4a701a77aa5293 Mon Sep 17 00:00:00 2001 From: Keegan McAllister Date: Thu, 4 Dec 2014 17:21:00 -0800 Subject: [PATCH 27/27] Add one more unresolved question --- text/0000-macro-reform.md | 3 +++ 1 file changed, 3 insertions(+) diff --git a/text/0000-macro-reform.md b/text/0000-macro-reform.md index e6fdf17b6a2..d2fab74d3e0 100644 --- a/text/0000-macro-reform.md +++ b/text/0000-macro-reform.md @@ -278,6 +278,9 @@ Should we forbid `$crate` in non-exported macros? It seems useless, however I think we should allow it anyway, to encourage the habit of writing `$crate::` for any references to the local crate. +Should `#[macro_reexport]` support the "glob" behavior of `#[macro_use]` with +no names listed? + # Acknowledgements This proposal is edited by Keegan McAllister. It has been refined through many