Support building with non-git repositories #95

denbeigh2000 · 2024-11-29T15:54:33Z

This is a repackaging of the work originally done in #80 and #82.

This allows embuild to build non-git source repositories. This work will allow dependencies (such was esp-idf) to be provided by sources other than embuild that may not include the surrounding git repo, such as the Nix package manager.

Reasonable testing has been completed in the comment below.

Closes #80
Closes #81
Closes #82

denbeigh2000 · 2024-12-01T19:59:16Z

Okay - I got a little caught out during testing while attempting to repro my failures in different scenarios, so I ended up writing a testing harness to get some repeatable results.

I'm extremely confident this change behaves in the way we expect it to, and I'm happy to perform testing in other scenarios, if desired.

Testing scenarios

building with `IDF_PATH` pointing to a non-git directory using current `embuild`/`esp-idf-sys` fails (testing current issue)

$ USE_LOCAL_IDF=1 MANGLE_GIT=1 USE_NEW_EMBUILD=0 bash end-to-end.sh
...

---

MANGLE_GIT is set to 1
Git will be removed from IDF_PATH

USE_LOCAL_IDF is set to 1
IDF_PATH will be set

USE_NEW_EMBUILD is set to 0
Forked embuild will not be used

---

...

   Compiling embuild v0.32.0   <-- using current embuild
   Compiling rustc-std-workspace-alloc v1.99.0 (/home/validation/.rustup/toolchains/esp/lib/rustlib/src/rust/library/rustc-std-workspace-alloc)
   Compiling panic_unwind v0.0.0 (/home/validation/.rustup/toolchains/esp/lib/rustlib/src/rust/library/panic_unwind)
   Compiling panic_abort v0.0.0 (/home/validation/.rustup/toolchains/esp/lib/rustlib/src/rust/library/panic_abort)
   Compiling gimli v0.29.0
   Compiling miniz_oxide v0.7.4
   Compiling object v0.36.2
   Compiling std_detect v0.1.5 (/home/validation/.rustup/toolchains/esp/lib/rustlib/src/rust/library/stdarch/crates/std_detect)
   Compiling hashbrown v0.14.5
   Compiling addr2line v0.22.0
   Compiling esp-idf-sys v0.35.0 (/home/validation/esp-idf-sys)
error: failed to run custom build command for `esp-idf-sys v0.35.0 (/home/validation/esp-idf-sys)`

Caused by:
  process didn't exit successfully: `/home/validation/esp-idf-sys/target/debug/build/esp-idf-sys-ba02850a5f182777/build-script-build` (exit status: 1)
  --- stdout
  cargo:rerun-if-env-changed=ESP_IDF_TOOLS_INSTALL_DIR
  cargo:rerun-if-env-changed=ESP_IDF_SDKCONFIG
  cargo:rerun-if-env-changed=ESP_IDF_SDKCONFIG_DEFAULTS
  cargo:rerun-if-env-changed=MCU
  cargo:rerun-if-env-changed=ESP_IDF_SYS_ROOT_CRATE
  cargo:rerun-if-env-changed=ESP_IDF_VERSION
  cargo:rerun-if-env-changed=ESP_IDF_REPOSITORY
  cargo:rerun-if-env-changed=ESP_IDF_CMAKE_GENERATOR
  cargo:rerun-if-env-changed=IDF_PATH
  cargo:rerun-if-env-changed=EXTRA-COMPONENTS
  cargo:rerun-if-env-changed=ESP_IDF_COMPONENTS
  cargo:rerun-if-env-changed=ESP_IDF_COMPONENT_MANAGER

  --- stderr
  Build configuration: BuildConfig {
      esp_idf_tools_install_dir: None,
      esp_idf_sdkconfig: None,
      esp_idf_sdkconfig_defaults: Some(
          [
              ".github/configs/sdkconfig.defaults",
          ],
      ),
      mcu: Some(
          "esp32",
      ),
      native: NativeConfig {
          esp_idf_version: Some(
              Tag(
                  "v5.3",
              ),
          ),
          esp_idf_repository: None,
          esp_idf_cmake_generator: None,
          idf_path: Some(
              "/home/validation/esp/esp-idf",   <-- using locally-managed esp-idf
          ),
          extra_components: [],
          esp_idf_components: None,
          esp_idf_component_manager: None,
      },
      esp_idf_sys_root_crate: None,
  }
  fatal: not a git repository (or any of the parent directories): .git
  fatal: not a git repository (or any of the parent directories): .git
  Error: '/home/validation/esp/esp-idf' is not a git respository

  Caused by:
      0: command 'cd "/home/validation/esp/esp-idf" && LANGUAGE="" LC_ALL="C.UTF-8" "git" "rev-parse" "--show-toplevel"' exited with non-zero status code 128
      1: fatal: not a git repository (or any of the parent directories): .git
warning: build failed, waiting for other jobs to finish...
Successfully copied 129MB to /home/denbeigh/esp-validation/build-dir-8VY
Copied build directory to /home/denbeigh/esp-validation/build-dir-8VY

building with `IDF_PATH` pointing to a non-git directory with updated `embuild`/`esp-idf-sys` passes (testing fix)

$ USE_LOCAL_IDF=1 MANGLE_GIT=1 USE_NEW_EMBUILD=1 bash end-to-end.sh
...

---

MANGLE_GIT is set to 1
Git will be removed from IDF_PATH

USE_LOCAL_IDF is set to 1
IDF_PATH will be set

USE_NEW_EMBUILD is set to 1
Forked embuild will be used

---

...

   Compiling envy v0.4.2
   Compiling embuild v0.32.0 (https://github.com/denbeigh2000/embuild?rev=b8b924b7a5bdbe947d5fff252cd24f61bda00074#b8b924b7)  <-- using my embuild
   Compiling cargo_metadata v0.18.1
   Compiling rustc-std-workspace-alloc v1.99.0 (/home/validation/.rustup/toolchains/esp/lib/rustlib/src/rust/library/rustc-std-workspace-alloc)
   Compiling panic_unwind v0.0.0 (/home/validation/.rustup/toolchains/esp/lib/rustlib/src/rust/library/panic_unwind)
   Compiling panic_abort v0.0.0 (/home/validation/.rustup/toolchains/esp/lib/rustlib/src/rust/library/panic_abort)
   Compiling gimli v0.29.0
   Compiling object v0.36.2
   Compiling miniz_oxide v0.7.4
   Compiling std_detect v0.1.5 (/home/validation/.rustup/toolchains/esp/lib/rustlib/src/rust/library/stdarch/crates/std_detect)
   Compiling hashbrown v0.14.5
   Compiling addr2line v0.22.0
   Compiling esp-idf-sys v0.35.0 (/home/validation/esp-idf-sys)
   Compiling proc_macro v0.0.0 (/home/validation/.rustup/toolchains/esp/lib/rustlib/src/rust/library/proc_macro)
   Compiling const_format v0.2.33
warning: unexpected `cfg` condition name: `esp_idf_version`
   --> examples/std_basics.rs:116:11
    |
116 | #[cfg(not(esp_idf_version = "4.3"))]
    |           ^^^^^^^^^^^^^^^^^^^^^^^
    |
    = help: expected names are: `clippy`, `debug_assertions`, `doc`, `docsrs`, `doctest`, `feature`, `fmt_debug`, `miri`, `overflow_checks`, `panic`, `proc_macro`, `relocation_model`, `rustfmt`, `sanitize`, `sanitizer_cfi_generalize_pointers`, `sanitizer_cfi_normalize_integers`, `target_abi`, `target_arch`, `target_endian`, `target_env`, `target_family`, `target_feature`, `target_has_atomic`, `target_has_atomic_equal_alignment`, `target_has_atomic_load_store`, `target_os`, `target_pointer_width`, `target_thread_local`, `target_vendor`, `test`, `ub_checks`, `unix`, and `windows`
    = help: consider using a Cargo feature instead
    = help: or consider adding in `Cargo.toml` the `check-cfg` lint config for the lint:
             [lints.rust]
             unexpected_cfgs = { level = "warn", check-cfg = ['cfg(esp_idf_version, values("4.3"))'] }
    = help: or consider adding `println!("cargo::rustc-check-cfg=cfg(esp_idf_version, values(\"4.3\"))");` to the top of the `build.rs`
    = note: see <https://doc.rust-lang.org/nightly/rustc/check-cfg/cargo-specifics.html> for more information about checking conditional configuration
    = note: `#[warn(unexpected_cfgs)]` on by default

warning: unexpected `cfg` condition name: `esp_idf_version`
  --> examples/std_basics.rs:31:15
   |
31 |     #[cfg(not(esp_idf_version = "4.3"))]
   |               ^^^^^^^^^^^^^^^^^^^^^^^
   |
   = help: consider using a Cargo feature instead
   = help: or consider adding in `Cargo.toml` the `check-cfg` lint config for the lint:
            [lints.rust]
            unexpected_cfgs = { level = "warn", check-cfg = ['cfg(esp_idf_version, values("4.3"))'] }
   = help: or consider adding `println!("cargo::rustc-check-cfg=cfg(esp_idf_version, values(\"4.3\"))");` to the top of the `build.rs`
   = note: see <https://doc.rust-lang.org/nightly/rustc/check-cfg/cargo-specifics.html> for more information about checking conditional configuration

warning: `esp-idf-sys` (example "std_basics") generated 2 warnings
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 1m 03s  <-- build success
~
Successfully copied 637MB to /home/denbeigh/esp-validation/build-dir-h1l
Copied build directory to /home/denbeigh/esp-validation/build-dir-h1l

$ cat build-dir-h1l/xtensa-esp32-espidf/debug/build/esp-idf-sys-4378245c83f62e3a/stderr
Build configuration: BuildConfig {
    esp_idf_tools_install_dir: None,
    esp_idf_sdkconfig: None,
    esp_idf_sdkconfig_defaults: Some(
        [
            ".github/configs/sdkconfig.defaults",
        ],
    ),
    mcu: Some(
        "esp32",
    ),
    native: NativeConfig {
        esp_idf_version: Some(
            Tag(
                "v5.3",
            ),
        ),
        esp_idf_repository: None,
        esp_idf_cmake_generator: None,
        idf_path: Some(
            "/home/validation/esp/esp-idf",  <-- using locally-managed esp-idf
        ),
        extra_components: [],
        esp_idf_components: None,
        esp_idf_component_manager: None,
    },
    esp_idf_sys_root_crate: None,
}
fatal: not a git repository (or any of the parent directories): .git
Using activated esp-idf v5.3.1 environment at '/home/validation/esp/esp-idf'
fatal: not a git repository (or any of the parent directories): .git
fatal: not a git repository (or any of the parent directories): .git
fatal: not a git repository: /home/validation/esp/esp-idf/components/openthread/openthread/../../../.git/modules/components/openthread/openthread
^^^ failures visible (non-fatal!)
Built components: xtensa, esp_driver_gpio, esp_pm, mbedtls, bootloader, esptool_py, partition_table, esp_app_format, esp_bootloader_format, app_update, esp_partition, efuse, bootloader_support, esp_mm, spi_flash, esp_system, esp_common, esp_rom, hal, log, heap, soc, esp_hw_support, freertos, newlib, pthread, cxx, esp_timer, esp_driver_gptimer, esp_ringbuf, esp_driver_uart, app_trace, esp_event, nvs_flash, esp_driver_pcnt, esp_driver_spi, esp_driver_mcpwm, esp_driver_ana_cmpr, esp_driver_i2s, sdmmc, esp_driver_sdmmc, esp_driver_sdspi, esp_driver_sdio, esp_driver_dac, esp_driver_rmt, esp_driver_tsens, esp_driver_sdm, esp_driver_i2c, esp_driver_ledc, esp_driver_parlio, esp_driver_usb_serial_jtag, driver, esp_phy, esp_vfs_console, vfs, lwip, esp_netif_stack, esp_netif, wpa_supplicant, esp_coex, esp_wifi, bt, unity, cmock, console, http_parser, esp-tls, esp_adc, esp_driver_isp, esp_driver_cam, esp_driver_jpeg, esp_driver_ppa, esp_driver_touch_sens, esp_eth, esp_gdbstub, esp_hid, tcp_transport, esp_http_client, esp_http_server, esp_https_ota, esp_https_server, esp_psram, esp_lcd, protobuf-c, protocomm, esp_local_ctrl, espcoredump, wear_levelling, fatfs, idf_test, ieee802154, json, mqtt, nvs_sec_provider, openthread, perfmon, spiffs, ulp, usb, wifi_provisioning, main
clang diag: /home/validation/esp/esp-idf/components/freertos/esp_additions/include/freertos/task_snapshot.h:8:2: warning: freertos/task_snapshot.h header is no longer used, and will be removed in future versions. [-W#warnings]
clang diag: /home/validation/esp/esp-idf/components/spi_flash/include/esp_spi_flash.h:6:2: warning: esp_spi_flash.h is deprecated, please use spi_flash_mmap.h instead [-W#warnings]
clang diag: /home/validation/esp/esp-idf/components/esp_adc/deprecated/include/esp_adc_cal.h:17:2: warning: "legacy adc calibration driver is deprecated, please migrate to use esp_adc/adc_cali.h and esp_adc/adc_cali_scheme.h" [-W#warnings]
clang diag: /home/validation/esp/esp-idf/components/wpa_supplicant/esp_supplicant/include/esp_wpa2.h:10:9: warning: esp_wpa2.h is deprecated. Use esp_eap_client.h instead. [-W#pragma-messages]
clang diag: /home/validation/esp/esp-idf/components/driver/deprecated/driver/adc.h:19:2: warning: "legacy adc driver is deprecated, please migrate to use esp_adc/adc_oneshot.h and esp_adc/adc_continuous.h for oneshot mode and continuous mode drivers respectively" [-W#warnings]
clang diag: /home/validation/esp/esp-idf/components/driver/deprecated/driver/dac.h:16:2: warning: "The legacy DAC driver is deprecated, please use `driver/dac_oneshot.h`, `driver/dac_cosine.h` or `driver/dac_continuous.h` instead" [-W#warnings]
clang diag: /home/validation/esp/esp-idf/components/driver/deprecated/driver/i2s.h:27:2: warning: "This set of I2S APIs has been deprecated, please include 'driver/i2s_std.h', 'driver/i2s_pdm.h' or 'driver/i2s_tdm.h' instead. if you want to keep using the old APIs and ignore this warning, you can enable 'Suppress legacy driver deprecated warning' option under 'I2S Configuration' menu in Kconfig" [-W#warnings]
clang diag: /home/validation/esp/esp-idf/components/driver/deprecated/driver/pcnt.h:15:2: warning: "legacy pcnt driver is deprecated, please migrate to use driver/pulse_cnt.h" [-W#warnings]
clang diag: /home/validation/esp/esp-idf/components/driver/deprecated/driver/periph_ctrl.h:7:2: warning: driver/periph_ctrl.h header is no longer used, and will be removed in future versions. [-W#warnings]
clang diag: /home/validation/esp/esp-idf/components/driver/deprecated/driver/rmt.h:18:2: warning: "The legacy RMT driver is deprecated, please use driver/rmt_tx.h and/or driver/rmt_rx.h" [-W#warnings]
clang diag: /home/validation/esp/esp-idf/components/driver/deprecated/driver/rtc_cntl.h:7:2: warning: driver/rtc_cntl.h header is no longer used, and will be removed in future versions. [-W#warnings]
clang diag: /home/validation/esp/esp-idf/components/driver/deprecated/driver/sigmadelta.h:15:2: warning: "The legacy sigma-delta driver is deprecated, please use driver/sdm.h" [-W#warnings]
clang diag: /home/validation/esp/esp-idf/components/driver/deprecated/driver/timer.h:16:2: warning: "legacy timer group driver is deprecated, please migrate to driver/gptimer.h" [-W#warnings]

building with `IDF_PATH` pointing to git directory with updated `embuild`/`esp-idf-sys` passes (testing no regression)

$ USE_LOCAL_IDF=1 MANGLE_GIT=0 USE_NEW_EMBUILD=1 bash end-to-end.sh
...

---

MANGLE_GIT is set to 0
Git will not be removed from IDF_PATH

USE_LOCAL_IDF is set to 1
IDF_PATH will be set

USE_NEW_EMBUILD is set to 1
Forked embuild will be used

---

...

   Compiling object v0.36.2
   Compiling miniz_oxide v0.7.4
   Compiling cargo_metadata v0.18.1
   Compiling embuild v0.32.0 (https://github.com/denbeigh2000/embuild?rev=b8b924b7a5bdbe947d5fff252cd24f61bda00074#b8b924b7)  <-- using my embuild
   Compiling addr2line v0.22.0
   Compiling esp-idf-sys v0.35.0 (/home/validation/esp-idf-sys)
   Compiling proc_macro v0.0.0 (/home/validation/.rustup/toolchains/esp/lib/rustlib/src/rust/library/proc_macro)
   Compiling const_format v0.2.33
warning: unexpected `cfg` condition name: `esp_idf_version`
   --> examples/std_basics.rs:116:11
    |
116 | #[cfg(not(esp_idf_version = "4.3"))]
    |           ^^^^^^^^^^^^^^^^^^^^^^^
    |
    = help: expected names are: `clippy`, `debug_assertions`, `doc`, `docsrs`, `doctest`, `feature`, `fmt_debug`, `miri`, `overflow_checks`, `panic`, `proc_macro`, `relocation_model`, `rustfmt`, `sanitize`, `sanitizer_cfi_generalize_pointers`, `sanitizer_cfi_normalize_integers`, `target_abi`, `target_arch`, `target_endian`, `target_env`, `target_family`, `target_feature`, `target_has_atomic`, `target_has_atomic_equal_alignment`, `target_has_atomic_load_store`, `target_os`, `target_pointer_width`, `target_thread_local`, `target_vendor`, `test`, `ub_checks`, `unix`, and `windows`
    = help: consider using a Cargo feature instead
    = help: or consider adding in `Cargo.toml` the `check-cfg` lint config for the lint:
             [lints.rust]
             unexpected_cfgs = { level = "warn", check-cfg = ['cfg(esp_idf_version, values("4.3"))'] }
    = help: or consider adding `println!("cargo::rustc-check-cfg=cfg(esp_idf_version, values(\"4.3\"))");` to the top of the `build.rs`
    = note: see <https://doc.rust-lang.org/nightly/rustc/check-cfg/cargo-specifics.html> for more information about checking conditional configuration
    = note: `#[warn(unexpected_cfgs)]` on by default

warning: unexpected `cfg` condition name: `esp_idf_version`
  --> examples/std_basics.rs:31:15
   |
31 |     #[cfg(not(esp_idf_version = "4.3"))]
   |               ^^^^^^^^^^^^^^^^^^^^^^^
   |
   = help: consider using a Cargo feature instead
   = help: or consider adding in `Cargo.toml` the `check-cfg` lint config for the lint:
            [lints.rust]
            unexpected_cfgs = { level = "warn", check-cfg = ['cfg(esp_idf_version, values("4.3"))'] }
   = help: or consider adding `println!("cargo::rustc-check-cfg=cfg(esp_idf_version, values(\"4.3\"))");` to the top of the `build.rs`
   = note: see <https://doc.rust-lang.org/nightly/rustc/check-cfg/cargo-specifics.html> for more information about checking conditional configuration

warning: `esp-idf-sys` (example "std_basics") generated 2 warnings
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 1m 05s   <-- build success
~
Successfully copied 637MB to /home/denbeigh/esp-validation/build-dir-dui
Copied build directory to /home/denbeigh/esp-validation/build-dir-dui

$ cat build-dir-dui/xtensa-esp32-espidf/debug/build/esp-idf-sys-4378245c83f62e3a/stderr
Build configuration: BuildConfig {
    esp_idf_tools_install_dir: None,
    esp_idf_sdkconfig: None,
    esp_idf_sdkconfig_defaults: Some(
        [
            ".github/configs/sdkconfig.defaults",
        ],
    ),
    mcu: Some(
        "esp32",
    ),
    native: NativeConfig {
        esp_idf_version: Some(
            Tag(
                "v5.3",
            ),
        ),
        esp_idf_repository: None,
        esp_idf_cmake_generator: None,
        idf_path: Some(
            "/home/validation/esp/esp-idf",  <-- using local idf checkout
        ),
        extra_components: [],
        esp_idf_components: None,
        esp_idf_component_manager: None,
    },
    esp_idf_sys_root_crate: None,
}
Using activated esp-idf v5.3.1 environment at '/home/validation/esp/esp-idf'
^^^ no git errors
Built components: xtensa, esp_driver_gpio, esp_pm, mbedtls, bootloader, esptool_py, partition_table, esp_app_format, esp_bootloader_format, app_update, esp_partition, efuse, bootloader_support, esp_mm, spi_flash, esp_system, esp_common, esp_rom, hal, log, heap, soc, esp_hw_support, freertos, newlib, pthread, cxx, esp_timer, esp_driver_gptimer, esp_ringbuf, esp_driver_uart, app_trace, esp_event, nvs_flash, esp_driver_pcnt, esp_driver_spi, esp_driver_mcpwm, esp_driver_ana_cmpr, esp_driver_i2s, sdmmc, esp_driver_sdmmc, esp_driver_sdspi, esp_driver_sdio, esp_driver_dac, esp_driver_rmt, esp_driver_tsens, esp_driver_sdm, esp_driver_i2c, esp_driver_ledc, esp_driver_parlio, esp_driver_usb_serial_jtag, driver, esp_phy, esp_vfs_console, vfs, lwip, esp_netif_stack, esp_netif, wpa_supplicant, esp_coex, esp_wifi, bt, unity, cmock, console, http_parser, esp-tls, esp_adc, esp_driver_isp, esp_driver_cam, esp_driver_jpeg, esp_driver_ppa, esp_driver_touch_sens, esp_eth, esp_gdbstub, esp_hid, tcp_transport, esp_http_client, esp_http_server, esp_https_ota, esp_https_server, esp_psram, esp_lcd, protobuf-c, protocomm, esp_local_ctrl, espcoredump, wear_levelling, fatfs, idf_test, ieee802154, json, mqtt, nvs_sec_provider, openthread, perfmon, spiffs, ulp, usb, wifi_provisioning, main
clang diag: /home/validation/esp/esp-idf/components/freertos/esp_additions/include/freertos/task_snapshot.h:8:2: warning: freertos/task_snapshot.h header is no longer used, and will be removed in future versions. [-W#warnings]
clang diag: /home/validation/esp/esp-idf/components/spi_flash/include/esp_spi_flash.h:6:2: warning: esp_spi_flash.h is deprecated, please use spi_flash_mmap.h instead [-W#warnings]
clang diag: /home/validation/esp/esp-idf/components/esp_adc/deprecated/include/esp_adc_cal.h:17:2: warning: "legacy adc calibration driver is deprecated, please migrate to use esp_adc/adc_cali.h and esp_adc/adc_cali_scheme.h" [-W#warnings]
clang diag: /home/validation/esp/esp-idf/components/wpa_supplicant/esp_supplicant/include/esp_wpa2.h:10:9: warning: esp_wpa2.h is deprecated. Use esp_eap_client.h instead. [-W#pragma-messages]
clang diag: /home/validation/esp/esp-idf/components/driver/deprecated/driver/adc.h:19:2: warning: "legacy adc driver is deprecated, please migrate to use esp_adc/adc_oneshot.h and esp_adc/adc_continuous.h for oneshot mode and continuous mode drivers respectively" [-W#warnings]
clang diag: /home/validation/esp/esp-idf/components/driver/deprecated/driver/dac.h:16:2: warning: "The legacy DAC driver is deprecated, please use `driver/dac_oneshot.h`, `driver/dac_cosine.h` or `driver/dac_continuous.h` instead" [-W#warnings]
clang diag: /home/validation/esp/esp-idf/components/driver/deprecated/driver/i2s.h:27:2: warning: "This set of I2S APIs has been deprecated, please include 'driver/i2s_std.h', 'driver/i2s_pdm.h' or 'driver/i2s_tdm.h' instead. if you want to keep using the old APIs and ignore this warning, you can enable 'Suppress legacy driver deprecated warning' option under 'I2S Configuration' menu in Kconfig" [-W#warnings]
clang diag: /home/validation/esp/esp-idf/components/driver/deprecated/driver/pcnt.h:15:2: warning: "legacy pcnt driver is deprecated, please migrate to use driver/pulse_cnt.h" [-W#warnings]
clang diag: /home/validation/esp/esp-idf/components/driver/deprecated/driver/periph_ctrl.h:7:2: warning: driver/periph_ctrl.h header is no longer used, and will be removed in future versions. [-W#warnings]
clang diag: /home/validation/esp/esp-idf/components/driver/deprecated/driver/rmt.h:18:2: warning: "The legacy RMT driver is deprecated, please use driver/rmt_tx.h and/or driver/rmt_rx.h" [-W#warnings]
clang diag: /home/validation/esp/esp-idf/components/driver/deprecated/driver/rtc_cntl.h:7:2: warning: driver/rtc_cntl.h header is no longer used, and will be removed in future versions. [-W#warnings]
clang diag: /home/validation/esp/esp-idf/components/driver/deprecated/driver/sigmadelta.h:15:2: warning: "The legacy sigma-delta driver is deprecated, please use driver/sdm.h" [-W#warnings]
clang diag: /home/validation/esp/esp-idf/components/driver/deprecated/driver/timer.h:16:2: warning: "legacy timer group driver is deprecated, please migrate to driver/gptimer.h" [-W#warnings]

building with `IDF_PATH` unset with updated `embuild`/`esp-idf-sys` passes (testing no regression)

$ USE_LOCAL_IDF=0 MANGLE_GIT=0 USE_NEW_EMBUILD=1 bash end-to-end.sh
...

---

MANGLE_GIT is set to 0
Git will not be removed from IDF_PATH

USE_LOCAL_IDF is set to 0
IDF_PATH will not be set

USE_NEW_EMBUILD is set to 1
Forked embuild will be used

---

...

   Compiling panic_unwind v0.0.0 (/home/validation/.rustup/toolchains/esp/lib/rustlib/src/rust/library/panic_unwind)
   Compiling cargo_metadata v0.18.1
   Compiling embuild v0.32.0 (https://github.com/denbeigh2000/embuild?rev=b8b924b7a5bdbe947d5fff252cd24f61bda00074#b8b924b7)  <-- using my embuild
   Compiling gimli v0.29.0
   Compiling miniz_oxide v0.7.4
   Compiling object v0.36.2
   Compiling std_detect v0.1.5 (/home/validation/.rustup/toolchains/esp/lib/rustlib/src/rust/library/stdarch/crates/std_detect)
   Compiling hashbrown v0.14.5
   Compiling addr2line v0.22.0
   Compiling esp-idf-sys v0.35.0 (/home/validation/esp-idf-sys)
   Compiling proc_macro v0.0.0 (/home/validation/.rustup/toolchains/esp/lib/rustlib/src/rust/library/proc_macro)
   Compiling const_format v0.2.33
warning: unexpected `cfg` condition name: `esp_idf_version`
   --> examples/std_basics.rs:116:11
    |
116 | #[cfg(not(esp_idf_version = "4.3"))]
    |           ^^^^^^^^^^^^^^^^^^^^^^^
    |
    = help: expected names are: `clippy`, `debug_assertions`, `doc`, `docsrs`, `doctest`, `feature`, `fmt_debug`, `miri`, `overflow_checks`, `panic`, `proc_macro`, `relocation_model`, `rustfmt`, `sanitize`, `sanitizer_cfi_generalize_pointers`, `sanitizer_cfi_normalize_integers`, `target_abi`, `target_arch`, `target_endian`, `target_env`, `target_family`, `target_feature`, `target_has_atomic`, `target_has_atomic_equal_alignment`, `target_has_atomic_load_store`, `target_os`, `target_pointer_width`, `target_thread_local`, `target_vendor`, `test`, `ub_checks`, `unix`, and `windows`
    = help: consider using a Cargo feature instead
    = help: or consider adding in `Cargo.toml` the `check-cfg` lint config for the lint:
             [lints.rust]
             unexpected_cfgs = { level = "warn", check-cfg = ['cfg(esp_idf_version, values("4.3"))'] }
    = help: or consider adding `println!("cargo::rustc-check-cfg=cfg(esp_idf_version, values(\"4.3\"))");` to the top of the `build.rs`
    = note: see <https://doc.rust-lang.org/nightly/rustc/check-cfg/cargo-specifics.html> for more information about checking conditional configuration
    = note: `#[warn(unexpected_cfgs)]` on by default

warning: unexpected `cfg` condition name: `esp_idf_version`
  --> examples/std_basics.rs:31:15
   |
31 |     #[cfg(not(esp_idf_version = "4.3"))]
   |               ^^^^^^^^^^^^^^^^^^^^^^^
   |
   = help: consider using a Cargo feature instead
   = help: or consider adding in `Cargo.toml` the `check-cfg` lint config for the lint:
            [lints.rust]
            unexpected_cfgs = { level = "warn", check-cfg = ['cfg(esp_idf_version, values("4.3"))'] }
   = help: or consider adding `println!("cargo::rustc-check-cfg=cfg(esp_idf_version, values(\"4.3\"))");` to the top of the `build.rs`
   = note: see <https://doc.rust-lang.org/nightly/rustc/check-cfg/cargo-specifics.html> for more information about checking conditional configuration

warning: `esp-idf-sys` (example "std_basics") generated 2 warnings
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 1m 53s  <-- build success
~
Successfully copied 636MB to /home/denbeigh/esp-validation/build-dir-yId
Copied build directory to /home/denbeigh/esp-validation/build-dir-yId

$ cat build-dir-yId/xtensa-esp32-espidf/debug/build/esp-idf-sys-4378245c83f62e3a/stderr
Build configuration: BuildConfig {
    esp_idf_tools_install_dir: None,
    esp_idf_sdkconfig: None,
    esp_idf_sdkconfig_defaults: Some(
        [
            ".github/configs/sdkconfig.defaults",
        ],
    ),
    mcu: Some(
        "esp32",
    ),
    native: NativeConfig {
        esp_idf_version: Some(
            Tag(
                "v5.1.2",
            ),
        ),
        esp_idf_repository: None,
        esp_idf_cmake_generator: None,
        idf_path: None,  <-- using self-managed IDF_PATH
        extra_components: [],
        esp_idf_components: None,
        esp_idf_component_manager: None,
    },
    esp_idf_sys_root_crate: None,
}
Using managed esp-idf repository: RemoteSdk { repo_url: None, git_ref: Tag("v5.1.2") }
Cloning into '/home/validation/esp-idf-sys/.embuild/espressif/esp-idf/v5.1.2'...  <-- embuild clones and configures idf-sys itself
Note: switching to '482a8fb2d78e3b58eb21b26da8a5bedf90623213'.

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by switching back to a branch.

If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -c with the switch command. Example:

  git switch -c <new-branch-name>

Or undo this operation with:

  git switch -

Turn off this advice by setting config variable advice.detachedHead to false

Submodule 'components/bootloader/subproject/components/micro-ecc/micro-ecc' (https://github.com/kmackay/micro-ecc.git) registered for path 'components/bootloader/subproject/components/micro-ecc/micro-ecc'
Submodule 'components/bt/controller/lib_esp32' (https://github.com/espressif/esp32-bt-lib.git) registered for path 'components/bt/controller/lib_esp32'
Submodule 'components/bt/controller/lib_esp32c2/esp32c2-bt-lib' (https://github.com/espressif/esp32c2-bt-lib.git) registered for path 'components/bt/controller/lib_esp32c2/esp32c2-bt-lib'
Submodule 'components/bt/controller/lib_esp32c3_family' (https://github.com/espressif/esp32c3-bt-lib.git) registered for path 'components/bt/controller/lib_esp32c3_family'
Submodule 'components/bt/controller/lib_esp32c6/esp32c6-bt-lib' (https://github.com/espressif/esp32c6-bt-lib.git) registered for path 'components/bt/controller/lib_esp32c6/esp32c6-bt-lib'
Submodule 'components/bt/controller/lib_esp32h2/esp32h2-bt-lib' (https://github.com/espressif/esp32h2-bt-lib.git) registered for path 'components/bt/controller/lib_esp32h2/esp32h2-bt-lib'
Submodule 'components/bt/host/nimble/nimble' (https://github.com/espressif/esp-nimble.git) registered for path 'components/bt/host/nimble/nimble'
Submodule 'components/cmock/CMock' (https://github.com/ThrowTheSwitch/CMock.git) registered for path 'components/cmock/CMock'
Submodule 'components/esp_coex/lib' (https://github.com/espressif/esp-coex-lib.git) registered for path 'components/esp_coex/lib'
Submodule 'components/esp_phy/lib' (https://github.com/espressif/esp-phy-lib.git) registered for path 'components/esp_phy/lib'
Submodule 'components/esp_wifi/lib' (https://github.com/espressif/esp32-wifi-lib.git) registered for path 'components/esp_wifi/lib'
Submodule 'components/heap/tlsf' (https://github.com/espressif/tlsf.git) registered for path 'components/heap/tlsf'
Submodule 'components/ieee802154/lib' (https://github.com/espressif/esp-ieee802154-lib.git) registered for path 'components/ieee802154/lib'
Submodule 'components/json/cJSON' (https://github.com/DaveGamble/cJSON.git) registered for path 'components/json/cJSON'
Submodule 'components/lwip/lwip' (https://github.com/espressif/esp-lwip.git) registered for path 'components/lwip/lwip'
Submodule 'components/mbedtls/mbedtls' (https://github.com/espressif/mbedtls.git) registered for path 'components/mbedtls/mbedtls'
Submodule 'components/mqtt/esp-mqtt' (https://github.com/espressif/esp-mqtt.git) registered for path 'components/mqtt/esp-mqtt'
Submodule 'components/openthread/lib' (https://github.com/espressif/esp-thread-lib.git) registered for path 'components/openthread/lib'
Submodule 'components/openthread/openthread' (https://github.com/espressif/openthread.git) registered for path 'components/openthread/openthread'
Submodule 'components/protobuf-c/protobuf-c' (https://github.com/protobuf-c/protobuf-c.git) registered for path 'components/protobuf-c/protobuf-c'
Submodule 'components/spiffs/spiffs' (https://github.com/pellepl/spiffs.git) registered for path 'components/spiffs/spiffs'
Submodule 'components/unity/unity' (https://github.com/ThrowTheSwitch/Unity.git) registered for path 'components/unity/unity'
Cloning into '/home/validation/esp-idf-sys/.embuild/espressif/esp-idf/v5.1.2/components/bootloader/subproject/components/micro-ecc/micro-ecc'...
Cloning into '/home/validation/esp-idf-sys/.embuild/espressif/esp-idf/v5.1.2/components/heap/tlsf'...
Cloning into '/home/validation/esp-idf-sys/.embuild/espressif/esp-idf/v5.1.2/components/esp_coex/lib'...
Cloning into '/home/validation/esp-idf-sys/.embuild/espressif/esp-idf/v5.1.2/components/bt/controller/lib_esp32'...
Cloning into '/home/validation/esp-idf-sys/.embuild/espressif/esp-idf/v5.1.2/components/json/cJSON'...
Cloning into '/home/validation/esp-idf-sys/.embuild/espressif/esp-idf/v5.1.2/components/bt/controller/lib_esp32c2/esp32c2-bt-lib'...
Cloning into '/home/validation/esp-idf-sys/.embuild/espressif/esp-idf/v5.1.2/components/ieee802154/lib'...
Cloning into '/home/validation/esp-idf-sys/.embuild/espressif/esp-idf/v5.1.2/components/bt/controller/lib_esp32h2/esp32h2-bt-lib'...
Cloning into '/home/validation/esp-idf-sys/.embuild/espressif/esp-idf/v5.1.2/components/bt/controller/lib_esp32c6/esp32c6-bt-lib'...
Cloning into '/home/validation/esp-idf-sys/.embuild/espressif/esp-idf/v5.1.2/components/bt/controller/lib_esp32c3_family'...
Cloning into '/home/validation/esp-idf-sys/.embuild/espressif/esp-idf/v5.1.2/components/bt/host/nimble/nimble'...
Cloning into '/home/validation/esp-idf-sys/.embuild/espressif/esp-idf/v5.1.2/components/lwip/lwip'...
Cloning into '/home/validation/esp-idf-sys/.embuild/espressif/esp-idf/v5.1.2/components/mqtt/esp-mqtt'...
Cloning into '/home/validation/esp-idf-sys/.embuild/espressif/esp-idf/v5.1.2/components/spiffs/spiffs'...
Cloning into '/home/validation/esp-idf-sys/.embuild/espressif/esp-idf/v5.1.2/components/openthread/lib'...
Cloning into '/home/validation/esp-idf-sys/.embuild/espressif/esp-idf/v5.1.2/components/protobuf-c/protobuf-c'...
Cloning into '/home/validation/esp-idf-sys/.embuild/espressif/esp-idf/v5.1.2/components/cmock/CMock'...
Cloning into '/home/validation/esp-idf-sys/.embuild/espressif/esp-idf/v5.1.2/components/esp_phy/lib'...
Cloning into '/home/validation/esp-idf-sys/.embuild/espressif/esp-idf/v5.1.2/components/unity/unity'...
Cloning into '/home/validation/esp-idf-sys/.embuild/espressif/esp-idf/v5.1.2/components/esp_wifi/lib'...
Cloning into '/home/validation/esp-idf-sys/.embuild/espressif/esp-idf/v5.1.2/components/mbedtls/mbedtls'...
Cloning into '/home/validation/esp-idf-sys/.embuild/espressif/esp-idf/v5.1.2/components/openthread/openthread'...
From https://github.com/kmackay/micro-ecc
 * branch            24c60e243580c7868f4334a1ba3123481fe1aa48 -> FETCH_HEAD
From https://github.com/espressif/esp32-bt-lib
 * branch            943b0f24eb6ed390213599e4a9556c66844ef0db -> FETCH_HEAD
From https://github.com/espressif/esp32c2-bt-lib
 * branch            ea33fbad1fa0879fe0e118359d20463b3e2f126b -> FETCH_HEAD
From https://github.com/espressif/esp32c3-bt-lib
 * branch            0cfac1b21ebc995e8e9aa040ab1ab29deee4f580 -> FETCH_HEAD
From https://github.com/espressif/esp32c6-bt-lib
 * branch            c32ea6a0c26025c1da8019d38216613686b4a139 -> FETCH_HEAD
From https://github.com/espressif/esp32h2-bt-lib
 * branch            e4a8094f283e4e477d47f590275f5b33d9c8cf74 -> FETCH_HEAD
From https://github.com/espressif/esp-nimble
 * branch            b85d99de41d8c9085c6f3de50a29ebedd9681381 -> FETCH_HEAD
From https://github.com/ThrowTheSwitch/CMock
 * branch            eeecc49ce8af123cf8ad40efdb9673e37b56230f -> FETCH_HEAD
Submodule 'vendor/c_exception' (https://github.com/throwtheswitch/cexception.git) registered for path 'components/cmock/CMock/vendor/c_exception'
Submodule 'vendor/unity' (https://github.com/throwtheswitch/unity.git) registered for path 'components/cmock/CMock/vendor/unity'
Cloning into '/home/validation/esp-idf-sys/.embuild/espressif/esp-idf/v5.1.2/components/cmock/CMock/vendor/c_exception'...
Cloning into '/home/validation/esp-idf-sys/.embuild/espressif/esp-idf/v5.1.2/components/cmock/CMock/vendor/unity'...
From https://github.com/throwtheswitch/cexception
 * branch            71b47be7c950f1bf5f7e5303779fa99a16224bb6 -> FETCH_HEAD
From https://github.com/throwtheswitch/unity
 * branch            cf949f45ca6d172a177b00da21310607b97bc7a7 -> FETCH_HEAD
From https://github.com/espressif/esp-coex-lib
 * branch            68cbbdb45cdc8c88ffbfc2587a4e7af020695e56 -> FETCH_HEAD
From https://github.com/espressif/esp-phy-lib
 * branch            a8e8b9532e2874ac167d4ade7808fda70fe05820 -> FETCH_HEAD
From https://github.com/espressif/esp32-wifi-lib
 * branch            4019e5e8ceafa862175b16b1e414c299502afc3f -> FETCH_HEAD
From https://github.com/espressif/tlsf
 * branch            8c9cd0517adf99e363812e9a295dfe3898fdd345 -> FETCH_HEAD
From https://github.com/DaveGamble/cJSON
 * branch            cb8693b058ba302f4829ec6d03f609ac6f848546 -> FETCH_HEAD
From https://github.com/espressif/esp-lwip
 * branch            7896c6cad020d17a986f7e850f603e084e319328 -> FETCH_HEAD
From https://github.com/espressif/mbedtls
 * branch            dd91f30058dfad68d80267cc82cd8472c4aa54b5 -> FETCH_HEAD
From https://github.com/espressif/esp-mqtt
 * branch            e6afdb4025fe018ae0add44e3c45249ea1974774 -> FETCH_HEAD
From https://github.com/espressif/esp-thread-lib
 * branch            12f563ee490236f7332eb22f568e71c7c1d4a3b7 -> FETCH_HEAD
From https://github.com/espressif/openthread
 * branch            af5938e389be40650507748272bb6c6b3a2de2cf -> FETCH_HEAD
From https://github.com/protobuf-c/protobuf-c
 * branch            abc67a11c6db271bedbb9f58be85d6f4e2ea8389 -> FETCH_HEAD
From https://github.com/pellepl/spiffs
 * branch            0dbb3f71c5f6fae3747a9d935372773762baf852 -> FETCH_HEAD
From https://github.com/ThrowTheSwitch/Unity
 * branch            7d2bf62b7e6afaf38153041a9d53c21aeeca9a25 -> FETCH_HEAD
Using esp-idf v5.1.2 at '/home/validation/esp-idf-sys/.embuild/espressif/esp-idf/v5.1.2'
Compiler supported targets: xtensa-esp32-elf

Built components: xtensa, esp_ringbuf, efuse, driver, esp_pm, mbedtls, esp_app_format, bootloader_support, bootloader, esptool_py, partition_table, esp_partition, app_update, esp_mm, spi_flash, pthread, esp_system, esp_rom, hal, log, heap, soc, esp_hw_support, freertos, newlib, cxx, esp_common, esp_timer, app_trace, esp_event, nvs_flash, esp_phy, vfs, lwip, esp_netif_stack, esp_netif, wpa_supplicant, esp_coex, esp_wifi, bt, unity, cmock, console, http_parser, esp-tls, esp_adc, esp_eth, esp_gdbstub, esp_hid, tcp_transport, esp_http_client, esp_http_server, esp_https_ota, esp_https_server, esp_psram, esp_lcd, protobuf-c, protocomm, esp_local_ctrl, espcoredump, wear_levelling, sdmmc, fatfs, idf_test, ieee802154, json, mqtt, openthread, perfmon, spiffs, ulp, usb, wifi_provisioning, main
clang diag: /home/validation/esp-idf-sys/.embuild/espressif/esp-idf/v5.1.2/components/spi_flash/include/esp_spi_flash.h:6:2: warning: esp_spi_flash.h is deprecated, please use spi_flash_mmap.h instead [-W#warnings]
clang diag: /home/validation/esp-idf-sys/.embuild/espressif/esp-idf/v5.1.2/components/esp_adc/deprecated/include/esp_adc_cal.h:17:2: warning: "legacy adc calibration driver is deprecated, please migrate to use esp_adc/adc_cali.h and esp_adc/adc_cali_scheme.h" [-W#warnings]
clang diag: /home/validation/esp-idf-sys/.embuild/espressif/esp-idf/v5.1.2/components/wpa_supplicant/esp_supplicant/include/esp_wpa2.h:10:9: warning: esp_wpa2.h is deprecated. Use esp_eap_client.h instead. [-W#pragma-messages]
clang diag: /home/validation/esp-idf-sys/.embuild/espressif/esp-idf/v5.1.2/components/driver/deprecated/driver/adc.h:19:2: warning: "legacy adc driver is deprecated, please migrate to use esp_adc/adc_oneshot.h and esp_adc/adc_continuous.h for oneshot mode and continuous mode drivers respectively" [-W#warnings]
clang diag: /home/validation/esp-idf-sys/.embuild/espressif/esp-idf/v5.1.2/components/driver/deprecated/driver/dac.h:16:2: warning: "The legacy DAC driver is deprecated, please use `driver/dac_oneshot.h`, `driver/dac_cosine.h` or `driver/dac_continuous.h` instead" [-W#warnings]
clang diag: /home/validation/esp-idf-sys/.embuild/espressif/esp-idf/v5.1.2/components/driver/deprecated/driver/i2s.h:27:2: warning: "This set of I2S APIs has been deprecated, please include 'driver/i2s_std.h', 'driver/i2s_pdm.h' or 'driver/i2s_tdm.h' instead. if you want to keep using the old APIs and ignore this warning, you can enable 'Suppress leagcy driver deprecated warning' option under 'I2S Configuration' menu in Kconfig" [-W#warnings]
clang diag: /home/validation/esp-idf-sys/.embuild/espressif/esp-idf/v5.1.2/components/driver/deprecated/driver/pcnt.h:15:2: warning: "legacy pcnt driver is deprecated, please migrate to use driver/pulse_cnt.h" [-W#warnings]
clang diag: /home/validation/esp-idf-sys/.embuild/espressif/esp-idf/v5.1.2/components/driver/deprecated/driver/periph_ctrl.h:7:2: warning: driver/periph_ctrl.h header is no longer used, and will be removed in future versions. [-W#warnings]
clang diag: /home/validation/esp-idf-sys/.embuild/espressif/esp-idf/v5.1.2/components/driver/deprecated/driver/rmt.h:18:2: warning: "The legacy RMT driver is deprecated, please use driver/rmt_tx.h and/or driver/rmt_rx.h" [-W#warnings]
clang diag: /home/validation/esp-idf-sys/.embuild/espressif/esp-idf/v5.1.2/components/driver/deprecated/driver/rtc_cntl.h:7:2: warning: driver/rtc_cntl.h header is no longer used, and will be removed in future versions. [-W#warnings]
clang diag: /home/validation/esp-idf-sys/.embuild/espressif/esp-idf/v5.1.2/components/driver/deprecated/driver/sigmadelta.h:15:2: warning: "The legacy sigma-delta driver is deprecated, please use driver/sdm.h" [-W#warnings]
clang diag: /home/validation/esp-idf-sys/.embuild/espressif/esp-idf/v5.1.2/components/driver/deprecated/driver/timer.h:16:2: warning: "legacy timer group driver is deprecated, please migrate to driver/gptimer.h" [-W#warnings]

ivmarkov

I think the PR is super-short and very clean, so I would like to apply it.

The one thing I'm struggling with is this external idf_path which might be passed from the outside, which kind of dillutes the semantics of EspIdf::from_env.

As in the other PR - can you elaborate why it is necessary, specifically in the context of using ESP IDF as a non-GIT repo?

src/espidf.rs

ivmarkov · 2024-12-22T15:23:25Z

Thank you for your patience!
I think the change is very nice and clean and I'll merge.
I'll look at what needs to be done on the esp-idf-sys side after that.

haileys and others added 10 commits November 24, 2024 19:04

add SourceTree enum in place of all direct git::Repository uses

77fc6ab

add SourceTree::Plain variant

380ac34

move SdkOrigin into espidf

efc4ffe

update is_managed_sdk docs to reflect current semantics

014e835

add SourceTree::open fn

81b8da8

make IDF_PATH_VAR relative to workspace in try_from_env

9bff898

update according to feedback

6b14fcf

fix clippy error

9d62661

cargo fmt

b8b924b

fix lint issues

be687ce

denbeigh2000 marked this pull request as draft November 30, 2024 21:20

denbeigh2000 marked this pull request as ready for review December 1, 2024 20:04

This was referenced Dec 1, 2024

embuild shouldn't require git or .git to exist in order to build the project #81

Closed

build cargo_driver: update embuild to support non-git IDF_PATH esp-rs/esp-idf-sys#353

Closed

ivmarkov requested changes Dec 2, 2024

View reviewed changes

src/espidf.rs Outdated Show resolved Hide resolved

src/espidf.rs Outdated Show resolved Hide resolved

split EspIdf::try_from_env into two functions

9d864a8

ivmarkov merged commit 051d7ce into esp-rs:master Dec 22, 2024
1 check passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Support building with non-git repositories #95

Support building with non-git repositories #95

denbeigh2000 commented Nov 29, 2024 •

edited

Loading

denbeigh2000 commented Dec 1, 2024

ivmarkov left a comment

ivmarkov commented Dec 22, 2024

Support building with non-git repositories #95

Support building with non-git repositories #95

Conversation

denbeigh2000 commented Nov 29, 2024 • edited Loading

denbeigh2000 commented Dec 1, 2024

Testing scenarios

ivmarkov left a comment

Choose a reason for hiding this comment

ivmarkov commented Dec 22, 2024

denbeigh2000 commented Nov 29, 2024 •

edited

Loading