Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Tracking issue for concat_idents #29599

Open
aturon opened this issue Nov 5, 2015 · 89 comments
Open

Tracking issue for concat_idents #29599

aturon opened this issue Nov 5, 2015 · 89 comments
Labels
A-macros Area: All kinds of macros (custom derive, macro_rules!, proc macros, ..) B-unstable Blocker: Implemented in the nightly compiler and unstable. C-tracking-issue Category: An issue tracking the progress of sth. like the implementation of an RFC Libs-Tracked Libs issues that are tracked on the team's project board. S-tracking-design-concerns Status: There are blocking design concerns. T-lang Relevant to the language team, which will review and decide on the PR/issue. T-libs-api Relevant to the library API team, which will review and decide on the PR/issue.

Comments

@aturon
Copy link
Member

aturon commented Nov 5, 2015

Tracks stabilization for the concat_idents macro.

Update(fmease): Please see #124225 (comment) (an alternative to this feature).

@aturon aturon added T-lang Relevant to the language team, which will review and decide on the PR/issue. B-unstable Blocker: Implemented in the nightly compiler and unstable. labels Nov 5, 2015
@retep998
Copy link
Member

retep998 commented Nov 5, 2015

concat_idents is so fundamentally useless at the moment because macros cannot be used in ident positions.

@ghost
Copy link

ghost commented Nov 5, 2015

Yeah it is pretty much useless right now (see #12249 #13294).

@nrc
Copy link
Member

nrc commented Nov 17, 2015

@eddyb
Copy link
Member

eddyb commented Nov 24, 2015

A solution that has been thrown around on IRC (but never ended up anywhere else AFAIK):
Have a macro invocation form for "early expansion", i.e. the following would work without macros in ident position:

macro_rules! wrap {
    ($name:ident) => { struct concat_idents!!(Wrapped, $name)($name); }
}

The "early expansion" macro_name!!(args) syntax was originally $*macro_name!(args) and is a subject of bikeshed.

If we move towards having all macros produce token streams that are parsed on-demand, such "early expansion" could be used in many more locations without adding macro support for each one.
Generic parameters, ADT fields, function signatures and match arms come to mind - there are so many recursive push-down hacks macros can end up with, just to construct lists of things and whatnot, that would simply not be necessary with "early expansion".

The only disadvantage is that concat_idents!! and friends would only work inside the RHS of macro_rules!, but I don't really see why you would need to use such a capability outside of a macro.

@skade
Copy link
Contributor

skade commented Dec 1, 2015

@skade
Copy link
Contributor

skade commented Dec 1, 2015

Well, given that mikkyang/rust-blas#12 is a nicer solution within the current language, I'd also like to raise my hand for "useless".

@nrc
Copy link
Member

nrc commented Mar 14, 2016

I think we should never stabilise the current version - as others have noted it is useless.

@aidanhs
Copy link
Member

aidanhs commented May 29, 2016

Since this is the tracking issue, other who stumble across it might be interested in https://github.com/SkylerLipthay/interpolate_idents which works on nightly rust.

@retep998
Copy link
Member

Unfortunately that solution will always depend on nightly Rust (until the day that syntax extensions become stable and pigs fly).

@skade
Copy link
Contributor

skade commented May 30, 2016

Given that the consensus seems to be "useless" and theres nicer solutions (e.g. [1], maybe we can remove this feature?

[1] https://github.com/mikkyang/rust-blas/pull/12/files

@eddyb
Copy link
Member

eddyb commented May 30, 2016

See rust-lang/rfcs#1628.

@aidanhs
Copy link
Member

aidanhs commented May 30, 2016

@retep998 yes, just noting it as a stopgap.

@skade it's the implementation rather than the idea that's useless. If concat_idents is removed rather than fixed, something else needs to fill its place (your solution doesn't work for all use-cases). I like the look of the rfc @eddyb linked, will follow that.

@skade
Copy link
Contributor

skade commented May 30, 2016

@aidanhs that still means that we should drop the feature to make sure no one uses it.

I agree that my solution doesn't cover all edge-cases, but all uses that I've currently seen in the wild. Also, but this is outside of my competence to decide, I think this is a perfect case for the use of a code generator.

@aidanhs
Copy link
Member

aidanhs commented Jun 1, 2016

@skade afaict nobody can use it, so that's not a big concern...but equally there's no reason to keep it around. My main motivation for leaving it was as a reminder that people do want the functionality, but I'm content to follow the RFC and have changed my position to "I don't mind either way".

Regarding use cases, here's the motivating example that brought me to this issue in the first place. Codegen's awesome for things like phf, but it feels a bit like overkill when I just want to generate some repetitive methods to extract tuples from enum variants. I guess it's down to personal preference.

@ExpHP
Copy link
Contributor

ExpHP commented Nov 24, 2017

An amusing fact about the current implementation of concat_idents! is that it will accept an empty argument list, making it possible to construct the empty string as an identifier:

error[E0425]: cannot find value `` in this scope
 --> src/main.rs:5:5
  |
5 |     concat_idents!();
  |     ^^^^^^^^^^^^^^^^^ not found in this scope

(good luck actually doing anything with it)

@petrochenkov
Copy link
Contributor

construct the empty string as an identifier

Empty string is used as a reserved identifier with special meaning in several places in the compiler, so this can potentially cause issues.

@jonhoo
Copy link
Contributor

jonhoo commented Jan 14, 2018

The linked RFC has been closed as postponed. Should concat_idents! now be removed? Is there any chance of seeing something in its place that supports generating identifiers?

@durka
Copy link
Contributor

durka commented Jan 15, 2018

The RFC should be reopened. It never should have been closed, as no better solution was proposed.

@skade
Copy link
Contributor

skade commented Jan 15, 2018

@durka I don't agree. There's ample reason to expect a new one when moving towards macros 2.0. RFCs are not necessarily closed to be replaced immediately, they are closed to stop following that line of discussion. Future RFCs might refer to it.

@alexreg
Copy link
Contributor

alexreg commented Jan 15, 2018

I'm with @durka on this.

@mimoo
Copy link

mimoo commented Oct 1, 2021

Is there a way to use this but with a type ty? Perhaps by extracting the ident from ty? But I can't find a way to do that in a macro_rules.

@ExpHP
Copy link
Contributor

ExpHP commented Oct 2, 2021

Once a macro_rules! macro parses a token as anything other than tt or ident it becomes completely opaque. If you need to do something more complicated like get the Foo from Foo<A, B> then you should consider writing a proc macro and using the syn and quote crates.

(or if it's a macro for internal use, you can always settle for some subset of type syntax that's easier to parse, e.g. $name:ident<$($param:ty),*>)

@joshtriplett joshtriplett added the S-tracking-design-concerns Status: There are blocking design concerns. label Nov 10, 2021
@joshtriplett
Copy link
Member

If we were to keep/stabilize this, it would need to work in the fashion people expect it to work (in place of an identifier).

@reitermarkus
Copy link
Contributor

I'd like for this to also support concatenating with integer literals, e.g.

const MY_CONST1: u32 = 1;
const MY_CONST2: u32 = 2;

concat_idents!(MY_CONST, 2)

@AlexisTM
Copy link

AlexisTM commented Feb 2, 2023

This is especially useful for embedded when we have many names we do not have control over. (register names) where most of the difference is a number.

@dvdsk
Copy link
Contributor

dvdsk commented Jun 1, 2023

It would be nice if we could use concat_idents! would optionally accept self. as first argument. This would allow accessing member variables and functions.

@reitermarkus
Copy link
Contributor

@dvdsk, I think this is already covered by #29599 (comment). If the macro works in ident positions, I assume you can simply use e.g. self.concat_idents!($field, 0).

@dvdsk
Copy link
Contributor

dvdsk commented Jun 1, 2023

@dvdsk, I think this is already covered by #29599 (comment). If the macro works in ident positions, I assume you can simply use e.g. self.concat_idents!($field, 0).

Ah I thought 'ident positions' referres to things like fn concat_idents!(macro_args)(function args) { ... }. Of course self can also be followed by an ident so its also an ident position.

Reading the rust reference the dot symbol (.) can currently only be followed by Field access or Tuple index. Not by a macro invocation. I think macro invocations need to be preceded by whitespace, I am no expect on Rust syntax however.

On a separate note self.macro might be confusing, one might think that syntax means calling a member macro? I would propose concat_idents!(self, test) resolving to self.test.

Finally self is not an ident but a keyword. So strictly speaking concat_idents should not cover it. So maybe we should drop concat_idents! and add a concat macro or look at #111930.

@tgross35
Copy link
Contributor

tgross35 commented Oct 8, 2023

Three comments:

  1. What are the problems with stabilizing this as-is and later adding the expected features? This does not seem to conflict anywhere.
  2. Are there problems with special casing this macro to allow only its use in the identifier position? Even only within other macros, so that the relationship between macro_rules! and concat_idents! is about the same as paste! and paste's [<...>] syntax. Obviously this would not be trivial to implement...
  3. This should be able to unstringify string literals, concat_idents!(foo, "bar") -> foobar (as paste does), for macros where it is logical for a user to specify a string but an identifier needs to be created from it

@sylv256
Copy link

sylv256 commented Nov 2, 2023

From the docs:

Also, as a general rule, macros are only allowed in item, statement or expression position. That means while you may use this macro for referring to existing variables, functions or modules etc, you cannot define a new one with it.

Is there any particular rationale for disallowing identifier concatenation in the definition of items? I really hate using paste.

@SimonSapin
Copy link
Contributor

It’s not that they’re excluded explicitly. It’s the other way around: parts of the language that expect an item, statement, or expression each accept various kinds of syntax, one of which being a macro invocation. In parts of the language that expect an identifier, the syntax is more restrictive (only accepting a plain or raw identifier token).

Accepting a macro invocation everywhere an identifier is accepted would be tricky at best because macro invocations themselves start with an identifier. paste adds dedicated [<>] syntax specifically to avoid these ambiguities. Maybe accepting a macro invocation in a few more specific places like the name of an fn item is possible, I don’t know. I suspect even coming up with a concrete and plausible proposal would take some work.

@DemiMarie
Copy link
Contributor

What about having concat_idents be a reserved word in Rust 2024? That would eliminate the ambiguity, because concat_idents! would no longer be a macro. Instead, it would be dedicated syntax that merely appeared to be a macro.

Of course, this might cause parsing problems. In particular, I doubt it would be compatible with an LALR(1) or LR(1) grammar.

@petrochenkov
Copy link
Contributor

At this point it would probably be better to implement ident concatenation as a "metavariable expression", like ${concat_idents(a, b, c)} or something.

Then in a code produced by macro_rules and parsed by Rust parser it will already be represented as a single identifier, so it will work in any position.

@c410-f3r
Copy link
Contributor

c410-f3r commented Nov 2, 2023

I think it would be nice to have something like ${concat_idents(a, ${concat_idents(b, c)}), i.e., a mechanism to eagerly evaluate a metavariable expression.

@tgross35
Copy link
Contributor

I'm sure there is a good answer but I don't see it here - why do we need concat_idents as a standalone macro in the first place? In theory it should be possible to have macro bodies have a syntax that concatenates things, like foo_##bar in C macros or [<foo_ $bar>] in paste. I guess this is in line with the metavar idea.

Also, it is extremely helpful to be able to convert between upper, lower, snake, and camel cases identifiers in paste. I sure wouldn't mind having this in std somehow if there were a good syntax...

bors added a commit to rust-lang-ci/rust that referenced this issue Jun 14, 2024
Add a new concat metavar expr

Revival of rust-lang#111930

Giving it another try now that rust-lang#117050 was merged.

With the new rules, meta-variable expressions must be referenced with a dollar sign (`$`) and this can cause misunderstands with `$concat`.

```rust
macro_rules! foo {
    ( $bar:ident ) => {
        const ${concat(VAR, bar)}: i32 = 1;
    };
}

// Will produce `VARbar` instead of `VAR_123`
foo!(_123);
```

In other words, forgetting the dollar symbol can produce undesired outputs.

cc rust-lang#29599
cc rust-lang#124225
@safinaskar
Copy link
Contributor

safinaskar commented Oct 13, 2024

🚀 I found a way to evaluate concat_idents (and concat and few other built-in macros) before evaluating other macro, which takes concat_idents as an argument! I. e. I found a way to evaluate a!(concat_idents!(...)) such way, that concat_idents evaluates before a. Answer is crate https://crates.io/crates/with_builtin_macros !!! In other words, with_builtin_macros is paste, but not only for concat_idents, but also for concat and some other macros.

And in other words, with_builtin_macros allows one to achieve eager evaluation of macros in limited way.

Also, with_builtin_macros allows one to use concat_idents when defining new identifier.

Also, https://crates.io/crates/with_builtin_macros allows one to use concat_idents in stable Rust.

// (This code was not tested, may contain typos)

fn concat_idents!(a, b) () {} // Doesn't work

with_builtin_macros::with_eager_expansions! {
  fn #{ concat_idents!(a, b) } () {} // Works! Even on stable!
}

macro_rules! this_macro_accepts_ident {
  ($a:ident) => {}
}

this_macro_accepts_ident!(concat_idents!(a, b)); // Doesn't work, because "this_macro_accepts_ident" evaluates before "concat_idents"

with_builtin_macros::with_eager_expansions! {
  this_macro_accepts_ident!(#{ concat_idents!(a, b) }); // Works! Even on stable!
}

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-macros Area: All kinds of macros (custom derive, macro_rules!, proc macros, ..) B-unstable Blocker: Implemented in the nightly compiler and unstable. C-tracking-issue Category: An issue tracking the progress of sth. like the implementation of an RFC Libs-Tracked Libs issues that are tracked on the team's project board. S-tracking-design-concerns Status: There are blocking design concerns. T-lang Relevant to the language team, which will review and decide on the PR/issue. T-libs-api Relevant to the library API team, which will review and decide on the PR/issue.
Projects
None yet
Development

No branches or pull requests