CD between reth_payload_builder and reth_node_api #6258

DoTheBestToGetTheBest · 2024-01-27T22:18:37Z

Describe the bug

I've identified a cyclic dependency between

reth_payload_builder and reth_node_api
reth_provider and reth_node_api
reth_stages and reth_blockchain_tree
reth_trie and reth_provider
crates, which is preventing successful compilation.

# reth-node-api/Cargo.toml
[dev-dependencies]
# for examples
reth-payload-builder.workspace = true

# reth-payload-builder/Cargo.toml
[dependencies]
...
reth-node-api.workspace = true

We should eliminate cd. We could refactor code, moving functionality between crates, or introducing new crates to break the cycles

I'm unable to run reth on wsl2 and unable to use reth crate. Also rust analyzer keep warning so it's impossible to do anything

Steps to reproduce

having wsl2 installed and rust analyzer

Node logs

no log

Platform(s)

Windows (x86)

What version/commit are you on?

main

What database version are you on?

none

What type of node are you running?

Archive (default)

What prune config do you use, if any?

none

If you've built Reth from source, provide the full command you used

From source.

Code of Conduct

I agree to follow the Code of Conduct

The text was updated successfully, but these errors were encountered:

Rjected · 2024-01-27T23:01:37Z

Do you mind sharing which command is failing and the logs from that? The cyclic dependencies are there only for examples, which compile fine as far as I remember

DoTheBestToGetTheBest · 2024-01-27T23:04:04Z

Do you mind sharing which command is failing and the logs from that? The cyclic dependencies are there only for examples, which compile fine as far as I remember

Sure

[ERROR project_model::workspace] cyclic deps: reth_auto_seal_consensus(Idx::<CrateData>(516)) -> reth(Idx::<CrateData>(513)), alternative path: reth(Idx::<CrateData>(513)) -> reth_auto_seal_consensus(Idx::<CrateData>(516))
[ERROR project_model::workspace] cyclic deps: reth_network(Idx::<CrateData>(558)) -> reth_network(Idx::<CrateData>(558)), alternative path: reth_network(Idx::<CrateData>(558))
[ERROR project_model::workspace] cyclic deps: reth_payload_builder(Idx::<CrateData>(568)) -> reth_node_api(Idx::<CrateData>(563)), alternative path: reth_node_api(Idx::<CrateData>(563)) -> reth_payload_builder(Idx::<CrateData>(568))
[ERROR project_model::workspace] cyclic deps: reth_provider(Idx::<CrateData>(574)) -> reth_node_api(Idx::<CrateData>(563)), alternative path: reth_node_api(Idx::<CrateData>(563)) -> reth_payload_builder(Idx::<CrateData>(568)) -> reth_provider(Idx::<CrateData>(574))
[ERROR project_model::workspace] cyclic deps: reth_stages(Idx::<CrateData>(588)) -> reth_blockchain_tree(Idx::<CrateData>(521)), alternative path: reth_blockchain_tree(Idx::<CrateData>(521)) -> reth_stages(Idx::<CrateData>(588))
[ERROR project_model::workspace] cyclic deps: reth_trie(Idx::<CrateData>(598)) -> reth_provider(Idx::<CrateData>(574)), alternative path: reth_provider(Idx::<CrateData>(574)) -> reth_trie(Idx::<CrateData>(598))
[ERROR project_model::workspace] cyclic deps: reth_auto_seal_consensus(Idx::<CrateData>(516)) -> reth(Idx::<CrateData>(513)), alternative path: reth(Idx::<CrateData>(513)) -> reth_auto_seal_consensus(Idx::<CrateData>(516))
[ERROR project_model::workspace] cyclic deps: reth_network(Idx::<CrateData>(558)) -> reth_network(Idx::<CrateData>(558)), alternative path: reth_network(Idx::<CrateData>(558))
[ERROR project_model::workspace] cyclic deps: reth_payload_builder(Idx::<CrateData>(568)) -> reth_node_api(Idx::<CrateData>(563)), alternative path: reth_node_api(Idx::<CrateData>(563)) -> reth_payload_builder(Idx::<CrateData>(568))
[ERROR project_model::workspace] cyclic deps: reth_provider(Idx::<CrateData>(574)) -> reth_node_api(Idx::<CrateData>(563)), alternative path: reth_node_api(Idx::<CrateData>(563)) -> reth_payload_builder(Idx::<CrateData>(568)) -> reth_provider(Idx::<CrateData>(574))
[ERROR project_model::workspace] cyclic deps: reth_stages(Idx::<CrateData>(588)) -> reth_blockchain_tree(Idx::<CrateData>(521)), alternative path: reth_blockchain_tree(Idx::<CrateData>(521)) -> reth_stages(Idx::<CrateData>(588))
[ERROR project_model::workspace] cyclic deps: reth_trie(Idx::<CrateData>(598)) -> reth_provider(Idx::<CrateData>(574)), alternative path: reth_provider(Idx::<CrateData>(574)) -> reth_trie(Idx::<CrateData>(598))
[ERROR rust_analyzer::main_loop] FetchBuildDataError:
error: failed to run custom build command for `libffi-sys v2.3.0`
note: To improve backtraces for build dependencies, set the CARGO_PROFILE_DEV_BUILD_OVERRIDE_DEBUG=true environment variable to enable debug information generation.

Caused by:
  process didn't exit successfully: `reth/target/debug/build/libffi-sys-ddce524bb04d37cd/build-script-build` (exit status: 101)
  --- stdout
  cargo:rerun-if-env-changed=CC_x86_64-unknown-linux-gnu
  CC_x86_64-unknown-linux-gnu = None
  cargo:rerun-if-env-changed=CC_x86_64_unknown_linux_gnu
  CC_x86_64_unknown_linux_gnu = None
  cargo:rerun-if-env-changed=HOST_CC
  HOST_CC = None
  cargo:rerun-if-env-changed=CC
  CC = None
  cargo:rerun-if-env-changed=CRATE_CC_NO_DEFAULTS
  CRATE_CC_NO_DEFAULTS = None
  CARGO_CFG_TARGET_FEATURE = Some("fxsr,sse,sse2")
  cargo:rerun-if-env-changed=CFLAGS_x86_64-unknown-linux-gnu
  CFLAGS_x86_64-unknown-linux-gnu = None
  cargo:rerun-if-env-changed=CFLAGS_x86_64_unknown_linux_gnu
  CFLAGS_x86_64_unknown_linux_gnu = None
  cargo:rerun-if-env-changed=HOST_CFLAGS
  HOST_CFLAGS = None
  cargo:rerun-if-env-changed=CFLAGS
  CFLAGS = None
  checking build system type... x86_64-pc-linux-gnu
  checking host system type... x86_64-pc-linux-gnu
  checking target system type... x86_64-pc-linux-gnu
  continue configure in default builddir "./x86_64-unknown-linux-gnu"
  ....exec /bin/bash .././configure "--srcdir=.." "--enable-builddir=x86_64-unknown-linux-gnu" "linux
  gnu"

  --- stderr
  .././configure: line 2247: config.log: No such file or directory
  .././configure: line 2257: config.log: No such file or directory
  cat: standard output: No such file or directory
  thread 'main' panicked at /home/user/.cargo/registry/src/index.crates.io-6f17d22bba15001f/libffi-sys-2.3.0/build/common.rs:8:5:
  Configuring libffi
  stack backtrace:
     0: rust_begin_unwind
               at /rustc/62d7ed4a6775c4490e493093ca98ef7c215b835b/library/std/src/panicking.rs:647:5
     1: core::panicking::panic_fmt
               at /rustc/62d7ed4a6775c4490e493093ca98ef7c215b835b/library/core/src/panicking.rs:72:14
     2: core::panicking::panic_display
     3: build_script_build::common::run_command
     4: build_script_build::not_msvc::configure_libffi
     5: build_script_build::not_msvc::build_and_link
     6: build_script_build::main
     7: core::ops::function::FnOnce::call_once
  note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.


[ERROR project_model::workspace] cyclic deps: reth_auto_seal_consensus(Idx::<CrateData>(516)) -> reth(Idx::<CrateData>(513)), alternative path: reth(Idx::<CrateData>(513)) -> reth_auto_seal_consensus(Idx::<CrateData>(516))
[ERROR project_model::workspace] cyclic deps: reth_network(Idx::<CrateData>(558)) -> reth_network(Idx::<CrateData>(558)), alternative path: reth_network(Idx::<CrateData>(558))
[ERROR project_model::workspace] cyclic deps: reth_payload_builder(Idx::<CrateData>(568)) -> reth_node_api(Idx::<CrateData>(563)), alternative path: reth_node_api(Idx::<CrateData>(563)) -> reth_payload_builder(Idx::<CrateData>(568))
[ERROR project_model::workspace] cyclic deps: reth_provider(Idx::<CrateData>(574)) -> reth_node_api(Idx::<CrateData>(563)), alternative path: reth_node_api(Idx::<CrateData>(563)) -> reth_payload_builder(Idx::<CrateData>(568)) -> reth_provider(Idx::<CrateData>(574))
[ERROR project_model::workspace] cyclic deps: reth_stages(Idx::<CrateData>(588)) -> reth_blockchain_tree(Idx::<CrateData>(521)), alternative path: reth_blockchain_tree(Idx::<CrateData>(521)) -> reth_stages(Idx::<CrateData>(588))
[ERROR project_model::workspace] cyclic deps: reth_trie(Idx::<CrateData>(598)) -> reth_provider(Idx::<CrateData>(574)), alternative path: reth_provider(Idx::<CrateData>(574)) -> reth_trie(Idx::<CrateData>(598))

Rjected · 2024-01-27T23:12:03Z

Do you mind sharing which command is failing and the logs from that? The cyclic dependencies are there only for examples, which compile fine as far as I remember

Sure

[ERROR project_model::workspace] cyclic deps: reth_auto_seal_consensus(Idx::<CrateData>(516)) -> reth(Idx::<CrateData>(513)), alternative path: reth(Idx::<CrateData>(513)) -> reth_auto_seal_consensus(Idx::<CrateData>(516))
[ERROR project_model::workspace] cyclic deps: reth_network(Idx::<CrateData>(558)) -> reth_network(Idx::<CrateData>(558)), alternative path: reth_network(Idx::<CrateData>(558))
[ERROR project_model::workspace] cyclic deps: reth_payload_builder(Idx::<CrateData>(568)) -> reth_node_api(Idx::<CrateData>(563)), alternative path: reth_node_api(Idx::<CrateData>(563)) -> reth_payload_builder(Idx::<CrateData>(568))
[ERROR project_model::workspace] cyclic deps: reth_provider(Idx::<CrateData>(574)) -> reth_node_api(Idx::<CrateData>(563)), alternative path: reth_node_api(Idx::<CrateData>(563)) -> reth_payload_builder(Idx::<CrateData>(568)) -> reth_provider(Idx::<CrateData>(574))
[ERROR project_model::workspace] cyclic deps: reth_stages(Idx::<CrateData>(588)) -> reth_blockchain_tree(Idx::<CrateData>(521)), alternative path: reth_blockchain_tree(Idx::<CrateData>(521)) -> reth_stages(Idx::<CrateData>(588))
[ERROR project_model::workspace] cyclic deps: reth_trie(Idx::<CrateData>(598)) -> reth_provider(Idx::<CrateData>(574)), alternative path: reth_provider(Idx::<CrateData>(574)) -> reth_trie(Idx::<CrateData>(598))
[ERROR project_model::workspace] cyclic deps: reth_auto_seal_consensus(Idx::<CrateData>(516)) -> reth(Idx::<CrateData>(513)), alternative path: reth(Idx::<CrateData>(513)) -> reth_auto_seal_consensus(Idx::<CrateData>(516))
[ERROR project_model::workspace] cyclic deps: reth_network(Idx::<CrateData>(558)) -> reth_network(Idx::<CrateData>(558)), alternative path: reth_network(Idx::<CrateData>(558))
[ERROR project_model::workspace] cyclic deps: reth_payload_builder(Idx::<CrateData>(568)) -> reth_node_api(Idx::<CrateData>(563)), alternative path: reth_node_api(Idx::<CrateData>(563)) -> reth_payload_builder(Idx::<CrateData>(568))
[ERROR project_model::workspace] cyclic deps: reth_provider(Idx::<CrateData>(574)) -> reth_node_api(Idx::<CrateData>(563)), alternative path: reth_node_api(Idx::<CrateData>(563)) -> reth_payload_builder(Idx::<CrateData>(568)) -> reth_provider(Idx::<CrateData>(574))
[ERROR project_model::workspace] cyclic deps: reth_stages(Idx::<CrateData>(588)) -> reth_blockchain_tree(Idx::<CrateData>(521)), alternative path: reth_blockchain_tree(Idx::<CrateData>(521)) -> reth_stages(Idx::<CrateData>(588))
[ERROR project_model::workspace] cyclic deps: reth_trie(Idx::<CrateData>(598)) -> reth_provider(Idx::<CrateData>(574)), alternative path: reth_provider(Idx::<CrateData>(574)) -> reth_trie(Idx::<CrateData>(598))
[ERROR rust_analyzer::main_loop] FetchBuildDataError:
error: failed to run custom build command for `libffi-sys v2.3.0`
note: To improve backtraces for build dependencies, set the CARGO_PROFILE_DEV_BUILD_OVERRIDE_DEBUG=true environment variable to enable debug information generation.

Caused by:
  process didn't exit successfully: `reth/target/debug/build/libffi-sys-ddce524bb04d37cd/build-script-build` (exit status: 101)
  --- stdout
  cargo:rerun-if-env-changed=CC_x86_64-unknown-linux-gnu
  CC_x86_64-unknown-linux-gnu = None
  cargo:rerun-if-env-changed=CC_x86_64_unknown_linux_gnu
  CC_x86_64_unknown_linux_gnu = None
  cargo:rerun-if-env-changed=HOST_CC
  HOST_CC = None
  cargo:rerun-if-env-changed=CC
  CC = None
  cargo:rerun-if-env-changed=CRATE_CC_NO_DEFAULTS
  CRATE_CC_NO_DEFAULTS = None
  CARGO_CFG_TARGET_FEATURE = Some("fxsr,sse,sse2")
  cargo:rerun-if-env-changed=CFLAGS_x86_64-unknown-linux-gnu
  CFLAGS_x86_64-unknown-linux-gnu = None
  cargo:rerun-if-env-changed=CFLAGS_x86_64_unknown_linux_gnu
  CFLAGS_x86_64_unknown_linux_gnu = None
  cargo:rerun-if-env-changed=HOST_CFLAGS
  HOST_CFLAGS = None
  cargo:rerun-if-env-changed=CFLAGS
  CFLAGS = None
  checking build system type... x86_64-pc-linux-gnu
  checking host system type... x86_64-pc-linux-gnu
  checking target system type... x86_64-pc-linux-gnu
  continue configure in default builddir "./x86_64-unknown-linux-gnu"
  ....exec /bin/bash .././configure "--srcdir=.." "--enable-builddir=x86_64-unknown-linux-gnu" "linux
  gnu"

  --- stderr
  .././configure: line 2247: config.log: No such file or directory
  .././configure: line 2257: config.log: No such file or directory
  cat: standard output: No such file or directory
  thread 'main' panicked at /home/user/.cargo/registry/src/index.crates.io-6f17d22bba15001f/libffi-sys-2.3.0/build/common.rs:8:5:
  Configuring libffi
  stack backtrace:
     0: rust_begin_unwind
               at /rustc/62d7ed4a6775c4490e493093ca98ef7c215b835b/library/std/src/panicking.rs:647:5
     1: core::panicking::panic_fmt
               at /rustc/62d7ed4a6775c4490e493093ca98ef7c215b835b/library/core/src/panicking.rs:72:14
     2: core::panicking::panic_display
     3: build_script_build::common::run_command
     4: build_script_build::not_msvc::configure_libffi
     5: build_script_build::not_msvc::build_and_link
     6: build_script_build::main
     7: core::ops::function::FnOnce::call_once
  note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.


[ERROR project_model::workspace] cyclic deps: reth_auto_seal_consensus(Idx::<CrateData>(516)) -> reth(Idx::<CrateData>(513)), alternative path: reth(Idx::<CrateData>(513)) -> reth_auto_seal_consensus(Idx::<CrateData>(516))
[ERROR project_model::workspace] cyclic deps: reth_network(Idx::<CrateData>(558)) -> reth_network(Idx::<CrateData>(558)), alternative path: reth_network(Idx::<CrateData>(558))
[ERROR project_model::workspace] cyclic deps: reth_payload_builder(Idx::<CrateData>(568)) -> reth_node_api(Idx::<CrateData>(563)), alternative path: reth_node_api(Idx::<CrateData>(563)) -> reth_payload_builder(Idx::<CrateData>(568))
[ERROR project_model::workspace] cyclic deps: reth_provider(Idx::<CrateData>(574)) -> reth_node_api(Idx::<CrateData>(563)), alternative path: reth_node_api(Idx::<CrateData>(563)) -> reth_payload_builder(Idx::<CrateData>(568)) -> reth_provider(Idx::<CrateData>(574))
[ERROR project_model::workspace] cyclic deps: reth_stages(Idx::<CrateData>(588)) -> reth_blockchain_tree(Idx::<CrateData>(521)), alternative path: reth_blockchain_tree(Idx::<CrateData>(521)) -> reth_stages(Idx::<CrateData>(588))
[ERROR project_model::workspace] cyclic deps: reth_trie(Idx::<CrateData>(598)) -> reth_provider(Idx::<CrateData>(574)), alternative path: reth_provider(Idx::<CrateData>(574)) -> reth_trie(Idx::<CrateData>(598))

what's the actual cargo command used to get the output here? Is this from an editor / IDE?

DoTheBestToGetTheBest · 2024-01-27T23:17:31Z

what's the actual cargo command used to get the output here? Is this from an editor / IDE?

as i said, i don't run any command cargo because rust analyzer is blocked by this so i can't use anything related to reth

DoTheBestToGetTheBest · 2024-01-28T18:18:30Z

@mattsse hey what do you suggest for this ?

Rjected · 2024-01-29T17:54:38Z

Looks like this is what is happening
rust-lang/rust-analyzer#14167

agree with one point from the comments:

Being able to have cycles in dev deps is an important bit of functionality for integration tests, doctests, examples, and benches. It does not have strange behavior there.

Pretty sure we would not be able to have examples in the node-api crate without a cyclic dep, regardless of how we shuffle crates around

DoTheBestToGetTheBest · 2024-01-29T20:04:19Z

Pretty sure we would not be able to have examples in the node-api crate without a cyclic dep, regardless of how we shuffle crates around

Thank you for your answer Rjected, i'm curious to know what could be the solution for this problem now ?

Do you have any idea?

DaniPopes · 2024-02-01T18:14:36Z

Is this an actual problem? Like what can you notice that doesn't work, where breaking the cycle would fix it? AFAIK this isn't an actual issue, since dev-dependencies are allowed to be circular

mattsse · 2024-02-01T18:21:05Z

yea I think @DoTheBestToGetTheBest only has these issues in RA, not familiar with that IDE so unclear if this is solvable with settings

DaniPopes · 2024-02-01T18:28:46Z

I do too, but I can't really notice anything

mattsse · 2024-02-01T18:30:19Z

in anycase this will be addressed by #6333

Degen-Player · 2024-02-01T22:10:50Z

Is this an actual problem? Like what can you notice that doesn't work, where breaking the cycle would fix it? AFAIK this isn't an actual issue, since dev-dependencies are allowed to be circular

I have same problem as OP #6128

Rust analyzer is probably the tool used by 70% of rust user it's not an IDE like said here

DoTheBestToGetTheBest · 2024-02-01T22:31:13Z

in anycase this will be addressed by #6333

Thank you guys for this fix !! Amzing from the part of reth team!

Blaeaea · 2024-02-02T07:02:58Z

Same problem with rust analyzer can't use reth anymore

Blaeaea · 2024-02-02T07:04:45Z

Not in WSL2 for my part in in macos

DoTheBestToGetTheBest added C-bug An unexpected or incorrect behavior S-needs-triage This issue needs to be labelled labels Jan 27, 2024

DoTheBestToGetTheBest mentioned this issue Feb 1, 2024

Add new helper trait for Network to get access to TransactionsHandle #5598

Closed

Rjected mentioned this issue Feb 1, 2024

Move payload builder primitive types to new crate #6333

Closed

DaniPopes removed the S-needs-triage This issue needs to be labelled label Feb 1, 2024

mattsse mentioned this issue Feb 5, 2024

feature: move node-api example into examples #6390

Merged

mattsse closed this as completed in #6390 Feb 5, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

CD between reth_payload_builder and reth_node_api #6258

CD between reth_payload_builder and reth_node_api #6258

DoTheBestToGetTheBest commented Jan 27, 2024

Rjected commented Jan 27, 2024

DoTheBestToGetTheBest commented Jan 27, 2024

Rjected commented Jan 27, 2024 •

edited

Loading

DoTheBestToGetTheBest commented Jan 27, 2024

DoTheBestToGetTheBest commented Jan 28, 2024

Rjected commented Jan 29, 2024

DoTheBestToGetTheBest commented Jan 29, 2024

DaniPopes commented Feb 1, 2024

mattsse commented Feb 1, 2024

DaniPopes commented Feb 1, 2024

mattsse commented Feb 1, 2024

Degen-Player commented Feb 1, 2024

DoTheBestToGetTheBest commented Feb 1, 2024

Blaeaea commented Feb 2, 2024

Blaeaea commented Feb 2, 2024

CD between reth_payload_builder and reth_node_api #6258

CD between reth_payload_builder and reth_node_api #6258

Comments

DoTheBestToGetTheBest commented Jan 27, 2024

Describe the bug

Steps to reproduce

Node logs

Platform(s)

What version/commit are you on?

What database version are you on?

What type of node are you running?

What prune config do you use, if any?

If you've built Reth from source, provide the full command you used

Code of Conduct

Rjected commented Jan 27, 2024

DoTheBestToGetTheBest commented Jan 27, 2024

Rjected commented Jan 27, 2024 • edited Loading

DoTheBestToGetTheBest commented Jan 27, 2024

DoTheBestToGetTheBest commented Jan 28, 2024

Rjected commented Jan 29, 2024

DoTheBestToGetTheBest commented Jan 29, 2024

DaniPopes commented Feb 1, 2024

mattsse commented Feb 1, 2024

DaniPopes commented Feb 1, 2024

mattsse commented Feb 1, 2024

Degen-Player commented Feb 1, 2024

DoTheBestToGetTheBest commented Feb 1, 2024

Blaeaea commented Feb 2, 2024

Blaeaea commented Feb 2, 2024

Rjected commented Jan 27, 2024 •

edited

Loading