Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feature std leaking when using macro in no_std env #47

Closed
koushiro opened this issue May 26, 2021 · 7 comments · Fixed by #48
Closed

feature std leaking when using macro in no_std env #47

koushiro opened this issue May 26, 2021 · 7 comments · Fixed by #48
Assignees
Labels

Comments

@koushiro
Copy link

koushiro commented May 26, 2021

In PR #35, data-encoding = { version = "2.3", default-features = false, features = ["alloc"] } was added into lib/macro/Cargo.toml and lib/macro/internal/Cargo.toml. It's ok for me.
But in the later commit 76e21dd, default-features = false, features = ["alloc"] was removed from lib/macro/internal/Cargo.toml, which cause the feature std of data-encoding leaking, I think.

Solution:

lib/macro/internal/Cargo.toml

[package]
name = "data-encoding-macro-internal"
version = "0.1.9"
authors = ["Julien Cretin <cretin@google.com>"]
license = "MIT"
edition = "2018"
description = "Internal library for data-encoding-macro"
readme = "README.md"
repository = "https://github.com/ia0/data-encoding"
include = ["Cargo.toml", "LICENSE", "README.md", "src/lib.rs"]

[lib]
proc-macro = true

[dependencies]
- data-encoding = { version = "2.3", path = "../.." }
+ data-encoding = { version = "2.3", path = "../..", default-features = false, features = ['alloc'] }
syn = "1"

BTW, we could add resolver = "2" to the Cargo.toml of downstream crate (only useful for rust 1.51+) to fix the issue (Cargo's New Feature Resolver), but we need to compile data-encoding twice.

@koushiro koushiro changed the title feature std leaking when using macro feature std leaking when using macro in no_std env May 26, 2021
@ia0
Copy link
Owner

ia0 commented May 26, 2021

Hi @koushiro

Thanks for the report!

As I see it, there are 2 problems:

  1. The fact that std may leak from data-encoding-macro-internal to data-encoding-macro. The recommended solution for that is to use the new resolver as described in the README. I actually forgot to publish it when I updated it. It's done now. Thanks for bringing that up!
  2. The data-encoding crate is built multiple times when using macros in no-std environments. I don't think this is fixable. There is one build with std for proc-macro and one build without. Note that there are only 2 builds when using alloc (and not 3). This is due to feature unification where the macro lib will use the alloc build instead of asking for a no-alloc build.

Does this clarify the situation?

@koushiro
Copy link
Author

koushiro commented May 26, 2021

@ia0 Thanks for your reply.

Is there a solution for earlier rust versions?

@ia0
Copy link
Owner

ia0 commented May 26, 2021

Yes, it's in the README too. I don't know the earliest version where it would work, but it's at least a year (tracking issue).

@koushiro
Copy link
Author

koushiro commented May 26, 2021

But it's doesn't work for earlier stable rust version, target thumbv6m-none-eabi.

you could try to run the below command under data-encoding/lib/macro directory:

cargo build --no-default-features --target thumbv6m-none-eabi

the target thumbv6m-none-eabi don't support libstd (only support liballoc and libcore), it's a good environment for no_std checking.

@ia0
Copy link
Owner

ia0 commented May 26, 2021

I'll look into this this week-end because it'll take some time. I want to do some experiments to see if there is a regression or if this is more of a feature request.

@ia0 ia0 self-assigned this May 26, 2021
ia0 added a commit that referenced this issue May 27, 2021
Fix #47. The problem is that for stable compilers older than 1.51, we used to
support no-std alloc environments, but not no-alloc environments. For compilers
1.51 and later, we additionally support no-alloc environments using resolver =
"2" in Cargo.toml of the end binary.
@ia0
Copy link
Owner

ia0 commented May 27, 2021

I could take a deeper look. I finally agree with your original post. Let me restate it with my own words:

  • After a89028b and before 76e21dd, it was possible to use the macro library with the alloc feature with stable compilers.
  • After 76e21dd, this was accidentally reverted in some refactoring.
  • This doesn't matter for compilers 1.51 and later, but it does for compilers 1.50 and earlier.
  • This can be fixed by using alloc feature in the macro internal proc-macro crate.
  • This means that for old stable compilers, only the alloc and std features can be used (currently only std is possible because of the bug). For newer compilers (stable and nightly) using the new resolver, the no-alloc feature can also be used.

However, note that it's still not clear to me how a project may use the alloc feature with a stable compiler. I have problems with #[alloc_error_handler].

@ia0 ia0 added the bug label May 27, 2021
@ia0 ia0 closed this as completed in #48 May 27, 2021
ia0 added a commit that referenced this issue May 27, 2021
Fix #47. The problem is that for stable compilers older than 1.51, we used to
support no-std alloc environments, but not no-alloc environments. For compilers
1.51 and later, we additionally support no-alloc environments using resolver =
"2" in Cargo.toml of the end binary.
@ia0
Copy link
Owner

ia0 commented May 27, 2021

I've published data-encoding-macro=0.1.12 (and data-encoding-macro-internal=0.1.10) which should fix your issue. Thanks again for reporting the regression!

hawkw added a commit to tosc-rs/mnemos that referenced this issue Aug 9, 2023
This commit changes the Cargo workspace setup to put all crates in One
Big Workspace, rather than having separate workspaces for some targets.
We now use the `per-package-target` unstable cargo feature to build
different crates for different targets. This means that `cargo` commands
in the root workspace now work without requiring the user to `cd` into a
particular directory to build a platform target --- for example, I can
now run:

```console
# in the repo root directory
$ cargo build -p mnemos-d1 --bin mq-pro
```

and build a MnemOS binary for the MQ Pro, without having to `cd` into
the MQ Pro directory.

One issue is that `cargo build --workspace` (and `check --workspace`,
etc) still does not work correctly, due to some [weird][1] [issues][2]
with feature unification which I don't entirely understand. However, as
a workaround, I've changed the workspace Cargo.toml to add a
[`default-members` field][3], so that running `cargo build` or `cargo
check` _without_ `--workspace` will build the subset of crates in the
`default-members` key. This way, `cargo {build, check, etc}` in the repo
root will do something reasonable by default, and the actual platform
targets can be built/checked with `cargo $WHATEVER --package $CRATE`.
IMO, this is still substantially nicer than having a bunch of separate
workspaces.

[1]: ia0/data-encoding#47
[2]: bincode-org/bincode#556
[3]: https://doc.rust-lang.org/cargo/reference/workspaces.html#the-default-members-field
hawkw added a commit to tosc-rs/mnemos that referenced this issue Aug 9, 2023
This commit changes the Cargo workspace setup to put all crates in One
Big Workspace, rather than having separate workspaces for some targets.
We now use the `per-package-target` unstable cargo feature to build
different crates for different targets. This means that `cargo` commands
in the root workspace now work without requiring the user to `cd` into a
particular directory to build a platform target --- for example, I can
now run:

```shell
# in the repo root directory
$ cargo build -p mnemos-d1 --bin mq-pro
```

and build a MnemOS binary for the MQ Pro, without having to `cd` into
the MQ Pro directory.

This is also necessary in order to make the `x86_64` build process added
in PR #216 work, since it relies on cargo artifact dependencies, which
appear not to work across workspaces.

One issue is that `cargo build --workspace` (and `check --workspace`,
etc) still does not work correctly, due to some [weird][1] [issues][2]
with feature unification which I don't entirely understand. However, as
a workaround, I've changed the workspace Cargo.toml to add a
[`default-members` field][3], so that running `cargo build` or `cargo
check`
_without_ `--workspace` will build the subset of crates in the
`default-members` key. This way, `cargo {build, check, etc}` in the repo
root will do something reasonable by default, and the actual platform
targets can be built/checked with `cargo $WHATEVER --package $CRATE`.
IMO, this is still substantially nicer than having a bunch of separate
workspaces.

[1]: ia0/data-encoding#47
[2]: bincode-org/bincode#556
[3]: https://doc.rust-lang.org/cargo/reference/workspaces.html#the-default-members-field
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants