Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Another attempt to support the rust language #19863

Merged
merged 5 commits into from
Mar 10, 2023
Merged

Conversation

lu-zero
Copy link
Contributor

@lu-zero lu-zero commented Nov 8, 2022

A rebase and then rework of #13916 with the current stable rust.

@lu-zero lu-zero force-pushed the master branch 2 times, most recently from 32031c8 to 7a7f52f Compare November 8, 2022 11:36
@lu-zero
Copy link
Contributor Author

lu-zero commented Nov 9, 2022

I removed the suricata package and just kept ripgrep as simpler testcase. Is there a way to locally run the CI tests to prevent further churn?

@trippleflux
Copy link

trippleflux commented Nov 10, 2022

@lu-zero
Thanks for the continuation, but how to use this?, currently trying to compile latest python-cryptography 38.0.3 but failed at :

make[2]: Entering directory '/home/username/works/OpenWrt/feeds/packages/lang/rust'
Makefile:180: WARNING: skipping rust -- package has no install section
make[2]: Nothing to be done for 'compile'.
make[2]: Leaving directory '/home/username/works/OpenWrt/feeds/packages/lang/rust'

Successfully installed cffi-1.15.0 pycparser-2.21
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
install -d -m0755 /home/username/works/OpenWrt/build_dir/target-arm_cortex-a15+neon-vfpv4_musl_eabi/pypi/cryptography-38.0.3/ipkg-install//usr/lib/python3.10/site-packages
cd "/home/username/works/OpenWrt/build_dir/target-arm_cortex-a15+neon-vfpv4_musl_eabi/pypi/cryptography-38.0.3/" && CC="ccache_cc" CCSHARED="ccache_cc -DPIC -fpic" CXX="ccache_cxx" LD="ccache_cc" LDSHARED="ccache_cc -shared" CFLAGS="-mtune=cortex-a15 -mfpu=neon-vfpv4 -O3 -funsafe-math-optimizations -ftree-vectorize -mtune=cortex-a15 -mfpu=neon-vfpv4 -O3 -funsafe-math-optimizations -ftree-vectorize -fhonour-copts -Wno-error=unused-but-set-variable -Wno-error=unused-result -mfloat-abi=hard -fmacro-prefix-map=/home/username/works/OpenWrt/build_dir/target-arm_cortex-a15+neon-vfpv4_musl_eabi/pypi/cryptography-38.0.3=cryptography-38.0.3 -Wformat -Werror=format-security -fstack-protector -D_FORTIFY_SOURCE=1 -Wl,-z,now -Wl,-z,relro" CPPFLAGS="-I/home/username/works/OpenWrt/staging_dir/toolchain-arm_cortex-a15+neon-vfpv4_gcc-12.2.0_musl_eabi/usr/include -I/home/username/works/OpenWrt/staging_dir/toolchain-arm_cortex-a15+neon-vfpv4_gcc-12.2.0_musl_eabi/include/fortify -I/home/username/works/OpenWrt/staging_dir/toolchain-arm_cortex-a15+neon-vfpv4_gcc-12.2.0_musl_eabi/include -I/home/username/works/OpenWrt/staging_dir/target-arm_cortex-a15+neon-vfpv4_musl_eabi/usr/include/python3.10" LDFLAGS="-L/home/username/works/OpenWrt/staging_dir/toolchain-arm_cortex-a15+neon-vfpv4_gcc-12.2.0_musl_eabi/usr/lib -L/home/username/works/OpenWrt/staging_dir/toolchain-arm_cortex-a15+neon-vfpv4_gcc-12.2.0_musl_eabi/lib -znow -zrelro -lpython3.10" _PYTHON_HOST_PLATFORM="linux-arm" __PYVENV_LAUNCHER__="/usr/bin/python3.10" PYTHONPATH="/home/username/works/OpenWrt/staging_dir/target-arm_cortex-a15+neon-vfpv4_musl_eabi/usr/lib/python3.10:/home/username/works/OpenWrt/staging_dir/target-arm_cortex-a15+neon-vfpv4_musl_eabi//usr/lib/python3.10/site-packages:/home/username/works/OpenWrt/build_dir/target-arm_cortex-a15+neon-vfpv4_musl_eabi/pypi/cryptography-38.0.3/ipkg-install//usr/lib/python3.10/site-packages" PYTHONDONTWRITEBYTECODE=1 PYTHONOPTIMIZE="" _python_sysroot="/home/username/works/OpenWrt/staging_dir/target-arm_cortex-a15+neon-vfpv4_musl_eabi" _python_prefix="/usr" _python_exec_prefix="/usr"     /home/username/works/OpenWrt/staging_dir/hostpkg/bin/python3.10  setup.py   install --prefix="/usr" --root="/home/username/works/OpenWrt/build_dir/target-arm_cortex-a15+neon-vfpv4_musl_eabi/pypi/cryptography-38.0.3/ipkg-install" --single-version-externally-managed

        =============================DEBUG ASSISTANCE==========================
        If you are seeing an error here please try the following to
        successfully install cryptography:

        Upgrade to the latest pip and try again. This will fix errors for most
        users. See: https://pip.pypa.io/en/stable/installing/#upgrading-pip
        =============================DEBUG ASSISTANCE==========================

Traceback (most recent call last):
  File "/home/username/works/OpenWrt/build_dir/target-arm_cortex-a15+neon-vfpv4_musl_eabi/pypi/cryptography-38.0.3/setup.py", line 17, in <module>
    from setuptools_rust import RustExtension
ModuleNotFoundError: No module named 'setuptools_rust'
make[2]: *** [Makefile:58: /home/username/works/OpenWrt/build_dir/target-arm_cortex-a15+neon-vfpv4_musl_eabi/pypi/cryptography-38.0.3/.built] Error 1
make[2]: Leaving directory '/home/username/works/OpenWrt/feeds/packages/lang/python/python-cryptography'
time: package/feeds/packages/python-cryptography/compile#3.73#2.73#9.12
    ERROR: package/feeds/packages/python-cryptography failed to build.
make[1]: *** [package/Makefile:116: package/feeds/packages/python-cryptography/compile] Error 1
make[1]: Leaving directory '/home/username/works/OpenWrt'
make: *** [/home/username/works/OpenWrt/include/toplevel.mk:231: package/python-cryptography/compile] Error 2

I am using default rust package compile option without addition .

[EDIT]
So currently it's host only?,

based on ripgrep package I have added the following into python-cryptography :

PKG_BUILD_DEPENDS:=libffi/host rust/host
include ../../../lang/rust/rust_environment.mk

And it goes with building rust.

Where is the suricata OpenWrt package now hosted?

[EDIT2]
Seems failed for arm_cortex-a15+neon-vfpv4_gcc-12.2.0_musl_eabi target:

Building stage2 std artifacts (x86_64-unknown-linux-gnu -> arm-openwrt-linux-muslgnueabi)
error: failed to run `rustc` to learn about target-specific information

Caused by:
  process didn't exit successfully: `/home/username/works/OpenWrt/build_dir/hostpkg/rust-1.65.0/build/bootstrap/debug/rustc - --crate-name ___ --print=file-names -Csymbol-mangling-version=legacy -Zunstable-options -Zunstable-options '--check-cfg=values(bootstrap)' '--check-cfg=values(stdarch_intel_sde)' '--check-cfg=values(no_fp_fmt_parse)' '--check-cfg=values(no_global_oom_handling)' '--check-cfg=values(freebsd12)' '--check-cfg=values(backtrace_in_libstd)' '--check-cfg=values(target_env,"libnx")' '--check-cfg=values(target_os,"watchos")' '--check-cfg=values(target_arch,"asmjs","spirv","nvptx","nvptx64","le32","xtensa")' '--check-cfg=values(dont_compile_me)' -Zmacro-backtrace -Clink-args=-Wl,-z,origin '-Clink-args=-Wl,-rpath,$ORIGIN/../lib' -Csplit-debuginfo=off -Cprefer-dynamic -L native=/home/username/works/OpenWrt/staging_dir/toolchain-arm_cortex-a15+neon-vfpv4_gcc-12.2.0_musl_eabi/lib -Cembed-bitcode=yes '-Zcrate-attr=doc(html_root_url="https://doc.rust-lang.org/1.65.0/")' --target arm-openwrt-linux-muslgnueabi --crate-type bin --crate-type rlib --crate-type dylib --crate-type cdylib --crate-type staticlib --crate-type proc-macro --print=sysroot --print=cfg` (exit status: 1)
  --- stderr
  error: Error loading target specification: Could not find specification for target "arm-openwrt-linux-muslgnueabi". Run `rustc --print target-list` for a list of built-in targets

Build completed unsuccessfully in 0:00:22
make[2]: *** [Makefile:180: /home/username/works/OpenWrt/build_dir/hostpkg/rust-1.65.0/.built] Error 1
make[2]: Leaving directory '/home/username/works/OpenWrt/feeds/packages/lang/rust'
time: package/feeds/packages/rust/host-compile#115.07#13.09#23.00
    ERROR: package/feeds/packages/rust [host] failed to build.
make[1]: *** [package/Makefile:114: package/feeds/packages/rust/host/compile] Error 1
make[1]: Leaving directory '/home/username/works/OpenWrt'
make: *** [/home/username/works/OpenWrt/include/toplevel.mk:231: package/python-cryptography/compile] Error 2

@lu-zero
Copy link
Contributor Author

lu-zero commented Nov 10, 2022

ModuleNotFoundError: No module named 'setuptools_rust'

Looks like it needs this package.

And also we should fix the host-mapping logic to support this case, which is the correct target in this case? arm-openwrt-linux-musl ?

@lu-zero
Copy link
Contributor Author

lu-zero commented Nov 10, 2022

The suricata packages are from this branch https://github.com/Itus-Shield/packages/commits/working

I use this toolchain for the aarch64-musl target to cross compile https://github.com/sifis-home/libp2p-rust-dht/ and the other packages being developed within sifis w/out problems :)

@trippleflux
Copy link

And also we should fix the host-mapping logic to support this case, which is the correct target in this case? arm-openwrt-linux-musl ?

Afaik cortex-a15 is armv7a, perhaps armv7-openwrt-linux-muslgnueabi would be enough?, I am not an experienced OpenWrt user or developer so please take precaution.

Currently I am edited manually on rust_environment.mk :

RUSTC_TARGET_ARCH:=armv7-openwrt-linux-muslgnueabi

And it compiled.

The suricata packages are from this branch https://github.com/Itus-Shield/packages/commits/working

I use this toolchain for the aarch64-musl target to cross compile https://github.com/sifis-home/libp2p-rust-dht/ and the other packages being developed within sifis w/out problems :)

Thanks, :-) It's about time suricata going OpenWrt 👍 , would be nice in x86_64. Also interested in learning rust, pretty cool language.

lang/rust/Makefile Outdated Show resolved Hide resolved
@trippleflux
Copy link

@lu-zero
Something wrong with cross compiling for arm_cortex-a15+neon-vfpv4_gcc-12.2.0_musl_eabi platform like I've mentioned above, the shared library seems compiled for the host (x86-64) as target, instead of arm-unknown-linux-musleabi or armv7-openwrt-linux-muslgnueabi :

root@username:/home/username/works/OpenWrt/build_dir/target-arm_cortex-a15+neon-vfpv4_musl_eabi/pypi/cryptography-38.0.3/.pkgdir/python3-cryptography/usr/lib/python3.10/site-packages/cryptography/hazmat/bindings# file _rust.abi3.so
_rust.abi3.so: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, BuildID[sha1]=4da699630436bad1a4d9e35c7aa5cc30d098589b, not stripped

My python-cryptography 38.0.3 draft Makefile :

#
# Copyright (C) 2015-2018 OpenWrt.org
#
# This is free software, licensed under the GNU General Public License v2.
# See /LICENSE for more information.
#

include $(TOPDIR)/rules.mk

PKG_NAME:=python-cryptography
PKG_VERSION:=38.0.3
PKG_RELEASE:=1

PYPI_NAME:=cryptography
PKG_HASH:=bfbe6ee19615b07a98b1d2287d6a6073f734735b49ee45b11324d85efc4d5cbd

PKG_LICENSE:=Apache-2.0 BSD-3-Clause
PKG_LICENSE_FILES:=LICENSE.APACHE LICENSE.BSD
PKG_MAINTAINER:=Jeffery To <jeffery.to@gmail.com>

PKG_BUILD_DEPENDS:=libffi/host rust/host python3/host

HOST_PYTHON3_PACKAGE_BUILD_DEPENDS:=cffi setuptools-rust # cffi>=1.12

include ../pypi.mk
include $(INCLUDE_DIR)/package.mk
include ../python3-package.mk
include ../../../lang/rust/rust_environment.mk

define Package/python3-cryptography
  SECTION:=lang
  CATEGORY:=Languages
  SUBMENU:=Python
  TITLE:=Cryptographic recipes and primitives
  URL:=https://github.com/pyca/cryptography
  DEPENDS:= \
      +libopenssl \
      +python3-light \
      +python3-email \
      +python3-openssl \
      +python3-urllib \
      +python3-cffi \
      +python3-six
endef

define Package/python3-cryptography/description
cryptography is a package which provides cryptographic recipes and
primitives to Python developers.  Our goal is for it to be your "cryptographic
standard library". It supports Python 2.6-2.7, Python 3.3+, and PyPy 2.6+.
endef

$(eval $(call Py3Package,python3-cryptography))
$(eval $(call BuildPackage,python3-cryptography))
$(eval $(call BuildPackage,python3-cryptography-src))

./feeds/packages/lang/python/host-pip-requirements/setuptools-rust.txt

setuptools-rust==1.5.2 --hash=sha256:d8daccb14dc0eae1b6b6eb3ecef79675bd37b4065369f79c35393dd5c55652c7
semantic-version==2.10.0 --hash=sha256:bdabb6d336998cbb378d4b9db3a4b56a1e3235701dc05ea2690d9a997ed5041c
typing-extensions==4.4.0 --hash=sha256:1511434bb92bf8dd198c12b1cc812e800d4181cfcb867674e0f8279cc93087aa

@trippleflux
Copy link

Going further after adding :

TARGET_LDFLAGS += \
		-L$(PYTHON3_LIB_DIR)
export PYO3_CROSS_INCLUDE_DIR=$(STAGING_DIR)/usr/include
export PYO3_CROSS_LIB_DIR=$(STAGING_DIR)/usr/lib
export CARGO_BUILD_TARGET=arm-unknown-linux-musleabi

into the python-cryptography Makefile

But still error on linking :

nown-linux-musleabi/release/deps/cryptography_rust.unindent-748906483c1c119e.unindent.e81da170-cgu.7.rcgu.o.rcgu.o" "/home/username/works/OpenWrt/build_dir/target-arm_cortex-a15+neon-vfpv4_musl_eabi/pypi/cryptography-38.0.3/src/rust/target/arm-unknown-linux-musleabi/release/deps/cryptography_rust.unindent-748906483c1c119e.unindent.e81da170-cgu.8.rcgu.o.rcgu.o" "/home/username/works/OpenWrt/build_dir/target-arm_cortex-a15+neon-vfpv4_musl_eabi/pypi/cryptography-38.0.3/src/rust/target/arm-unknown-linux-musleabi/release/deps/cryptography_rust.unindent-748906483c1c119e.unindent.e81da170-cgu.9.rcgu.o.rcgu.o" "/home/username/works/OpenWrt/build_dir/target-arm_cortex-a15+neon-vfpv4_musl_eabi/pypi/cryptography-38.0.3/src/rust/target/arm-unknown-linux-musleabi/release/deps/cryptography_rust.unwind-3c091545acb97779.unwind.2e387473-cgu.0.rcgu.o.rcgu.o" "-Wl,--as-needed" "-L" "/home/username/works/OpenWrt/build_dir/target-arm_cortex-a15+neon-vfpv4_musl_eabi/pypi/cryptography-38.0.3/src/rust/target/arm-unknown-linux-musleabi/release/deps" "-L" "/home/username/works/OpenWrt/build_dir/target-arm_cortex-a15+neon-vfpv4_musl_eabi/pypi/cryptography-38.0.3/src/rust/target/release/deps" "-L" "/home/username/works/OpenWrt/staging_dir/host/lib64/rustlib/arm-unknown-linux-musleabi/lib" "-Wl,-Bstatic" "/home/username/works/OpenWrt/staging_dir/host/lib/rustlib/arm-unknown-linux-musleabi/lib/libcompiler_builtins-a637be64e8863187.rlib" "-Wl,-Bdynamic" "-lgcc_s" "-lc" "-Wl,--eh-frame-hdr" "-Wl,-znoexecstack" "-L" "/home/username/works/OpenWrt/staging_dir/host/lib64/rustlib/arm-unknown-linux-musleabi/lib" "-o" "/home/username/works/OpenWrt/build_dir/target-arm_cortex-a15+neon-vfpv4_musl_eabi/pypi/cryptography-38.0.3/src/rust/target/arm-unknown-linux-musleabi/release/deps/libcryptography_rust.so" "-Wl,--gc-sections" "-shared" "-Wl,-zrelro,-znow" "-Wl,-O1" "-nodefaultlibs"
  = note: /usr/bin/ld: /home/username/works/OpenWrt/build_dir/target-arm_cortex-a15+neon-vfpv4_musl_eabi/pypi/cryptography-38.0.3/src/rust/target/arm-unknown-linux-musleabi/release/deps/cryptography_rust.5d0d1978mqpjhaxk.rcgu.o: relocations in generic ELF (EM: 40)
          /usr/bin/ld: /home/username/works/OpenWrt/build_dir/target-arm_cortex-a15+neon-vfpv4_musl_eabi/pypi/cryptography-38.0.3/src/rust/target/arm-unknown-linux-musleabi/release/deps/cryptography_rust.5d0d1978mqpjhaxk.rcgu.o: relocations in generic ELF (EM: 40)
          /usr/bin/ld: /home/username/works/OpenWrt/build_dir/target-arm_cortex-a15+neon-vfpv4_musl_eabi/pypi/cryptography-38.0.3/src/rust/target/arm-unknown-linux-musleabi/release/deps/cryptography_rust.5d0d1978mqpjhaxk.rcgu.o: relocations in generic ELF (EM: 40)
          /usr/bin/ld: /home/username/works/OpenWrt/build_dir/target-arm_cortex-a15+neon-vfpv4_musl_eabi/pypi/cryptography-38.0.3/src/rust/target/arm-unknown-linux-musleabi/release/deps/cryptography_rust.5d0d1978mqpjhaxk.rcgu.o: relocations in generic ELF (EM: 40)
          /usr/bin/ld: /home/username/works/OpenWrt/build_dir/target-arm_cortex-a15+neon-vfpv4_musl_eabi/pypi/cryptography-38.0.3/src/rust/target/arm-unknown-linux-musleabi/release/deps/cryptography_rust.5d0d1978mqpjhaxk.rcgu.o: relocations in generic ELF (EM: 40)
          /usr/bin/ld: /home/username/works/OpenWrt/build_dir/target-arm_cortex-a15+neon-vfpv4_musl_eabi/pypi/cryptography-38.0.3/src/rust/target/arm-unknown-linux-musleabi/release/deps/cryptography_rust.5d0d1978mqpjhaxk.rcgu.o: relocations in generic ELF (EM: 40)
          /usr/bin/ld: /home/username/works/OpenWrt/build_dir/target-arm_cortex-a15+neon-vfpv4_musl_eabi/pypi/cryptography-38.0.3/src/rust/target/arm-unknown-linux-musleabi/release/deps/cryptography_rust.5d0d1978mqpjhaxk.rcgu.o: relocations in generic ELF (EM: 40)
          /usr/bin/ld: /home/username/works/OpenWrt/build_dir/target-arm_cortex-a15+neon-vfpv4_musl_eabi/pypi/cryptography-38.0.3/src/rust/target/arm-unknown-linux-musleabi/release/deps/cryptography_rust.5d0d1978mqpjhaxk.rcgu.o: relocations in generic ELF (EM: 40)
          /usr/bin/ld: /home/username/works/OpenWrt/build_dir/target-arm_cortex-a15+neon-vfpv4_musl_eabi/pypi/cryptography-38.0.3/src/rust/target/arm-unknown-linux-musleabi/release/deps/cryptography_rust.5d0d1978mqpjhaxk.rcgu.o: relocations in generic ELF (EM: 40)
          /usr/bin/ld: /home/username/works/OpenWrt/build_dir/target-arm_cortex-a15+neon-vfpv4_musl_eabi/pypi/cryptography-38.0.3/src/rust/target/arm-unknown-linux-musleabi/release/deps/cryptography_rust.5d0d1978mqpjhaxk.rcgu.o: relocations in generic ELF (EM: 40)
          /usr/bin/ld: /home/username/works/OpenWrt/build_dir/target-arm_cortex-a15+neon-vfpv4_musl_eabi/pypi/cryptography-38.0.3/src/rust/target/arm-unknown-linux-musleabi/release/deps/cryptography_rust.5d0d1978mqpjhaxk.rcgu.o: error adding symbols: file in wrong format
          collect2: error: ld returned 1 exit status


error: could not compile `cryptography-rust` due to previous error

Caused by:
  process didn't exit successfully: `rustc --crate-name cryptography_rust --edition=2018 src/lib.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --crate-type cdylib --emit=dep-info,link -C opt-level=3 -C lto=thin --crate-type cdylib -C overflow-checks=on --cfg 'feature="default"' --cfg 'feature="extension-module"' -C metadata=ad553caa0913c0d5 --out-dir /home/username/works/OpenWrt/build_dir/target-arm_cortex-a15+neon-vfpv4_musl_eabi/pypi/cryptography-38.0.3/src/rust/target/arm-unknown-linux-musleabi/release/deps --target arm-unknown-linux-musleabi -L dependency=/home/username/works/OpenWrt/build_dir/target-arm_cortex-a15+neon-vfpv4_musl_eabi/pypi/cryptography-38.0.3/src/rust/target/arm-unknown-linux-musleabi/release/deps -L dependency=/home/username/works/OpenWrt/build_dir/target-arm_cortex-a15+neon-vfpv4_musl_eabi/pypi/cryptography-38.0.3/src/rust/target/release/deps --extern asn1=/home/username/works/OpenWrt/build_dir/target-arm_cortex-a15+neon-vfpv4_musl_eabi/pypi/cryptography-38.0.3/src/rust/target/arm-unknown-linux-musleabi/release/deps/libasn1-d4315fe5564406fe.rlib --extern chrono=/home/username/works/OpenWrt/build_dir/target-arm_cortex-a15+neon-vfpv4_musl_eabi/pypi/cryptography-38.0.3/src/rust/target/arm-unknown-linux-musleabi/release/deps/libchrono-90a85769a2b0139c.rlib --extern once_cell=/home/username/works/OpenWrt/build_dir/target-arm_cortex-a15+neon-vfpv4_musl_eabi/pypi/cryptography-38.0.3/src/rust/target/arm-unknown-linux-musleabi/release/deps/libonce_cell-3057188e9b074a7a.rlib --extern ouroboros=/home/username/works/OpenWrt/build_dir/target-arm_cortex-a15+neon-vfpv4_musl_eabi/pypi/cryptography-38.0.3/src/rust/target/arm-unknown-linux-musleabi/release/deps/libouroboros-bf5a111e901f024b.rlib --extern pem=/home/username/works/OpenWrt/build_dir/target-arm_cortex-a15+neon-vfpv4_musl_eabi/pypi/cryptography-38.0.3/src/rust/target/arm-unknown-linux-musleabi/release/deps/libpem-81f502f57eda75c2.rlib --extern pyo3=/home/username/works/OpenWrt/build_dir/target-arm_cortex-a15+neon-vfpv4_musl_eabi/pypi/cryptography-38.0.3/src/rust/target/arm-unknown-linux-musleabi/release/deps/libpyo3-9e81ad91923da1fd.rlib -Ctarget-feature=-crt-static` (exit status: 1)

    =============================DEBUG ASSISTANCE=============================
    If you are seeing a compilation error please try the following steps to
    successfully install cryptography:
    1) Upgrade to the latest pip and try again. This will fix errors for most
       users. See: https://pip.pypa.io/en/stable/installing/#upgrading-pip
    2) Read https://cryptography.io/en/latest/installation/ for specific
       instructions for your platform.
    3) Check our frequently asked questions for more information:
       https://cryptography.io/en/latest/faq/
    4) Ensure you have a recent Rust toolchain installed:
       https://cryptography.io/en/latest/installation/#rust

    Python: 3.10.7
    platform: Linux-6.0.0-3-amd64-x86_64-with-glibc2.36
    pip: 22.2.2
    setuptools: 63.2.0
    setuptools_rust: 1.5.2
    rustc: 1.65.0
    =============================DEBUG ASSISTANCE=============================

error: `cargo rustc --lib --message-format=json-render-diagnostics --manifest-path src/rust/Cargo.toml --target arm-unknown-linux-musleabi --release -v --features 'pyo3/extension-module pyo3/abi3-py36' -- --crate-type cdylib` failed with code 101
make[2]: *** [Makefile:65: /home/username/works/OpenWrt/build_dir/target-arm_cortex-a15+neon-vfpv4_musl_eabi/pypi/cryptography-38.0.3/.built] Error 1
make[2]: Leaving directory '/home/username/works/OpenWrt/feeds/packages/lang/python/python-cryptography'
time: package/feeds/packages/python-cryptography/compile#72.23#20.37#22.66
    ERROR: package/feeds/packages/python-cryptography failed to build.
make[1]: *** [package/Makefile:116: package/feeds/packages/python-cryptography/compile] Error 1
make[1]: Leaving directory '/home/username/works/OpenWrt'
make: *** [/home/username/works/OpenWrt/include/toplevel.mk:231: package/python-cryptography/compile] Error 2

arm_cortex-a15+neon-vfpv4_gcc-12.2.0_musl_eabi cpuinfo :

root@router:~# cat /proc/cpuinfo
processor       : 0
model name      : ARMv7 Processor rev 0 (v7l)
BogoMIPS        : 12.50
Features        : half thumb fastmult vfp edsp neon vfpv3 tls vfpv4 idiva idivt vfpd32
CPU implementer : 0x51
CPU architecture: 7
CPU variant     : 0x2
CPU part        : 0x04d
CPU revision    : 0

processor       : 1
model name      : ARMv7 Processor rev 0 (v7l)
BogoMIPS        : 26.04
Features        : half thumb fastmult vfp edsp neon vfpv3 tls vfpv4 idiva idivt vfpd32
CPU implementer : 0x51
CPU architecture: 7
CPU variant     : 0x2
CPU part        : 0x04d
CPU revision    : 0

Hardware        : Generic DT based system
Revision        : 0000
Serial          : 0000000000000000

just tested arm-unknown-linux-musleabi RUSTC_TARGET_ARCH is resulting :

root@router:~# readelf -A cryptography_rust.5d0d1978mqpjhaxk.rcgu.o
Attribute Section: aeabi
File Attributes
  Tag_conformance: "2.09"
  Tag_CPU_arch: v6
  Tag_ARM_ISA_use: Yes
  Tag_THUMB_ISA_use: Thumb-1
  Tag_ABI_PCS_R9_use: V6
  Tag_ABI_PCS_RW_data: PC-relative
  Tag_ABI_PCS_RO_data: PC-relative
  Tag_ABI_PCS_GOT_use: GOT-indirect
  Tag_ABI_FP_denormal: Needed
  Tag_ABI_FP_exceptions: Unused
  Tag_ABI_FP_number_model: IEEE 754
  Tag_ABI_align_needed: 8-byte
  Tag_ABI_align_preserved: 8-byte, except leaf SP
  Tag_ABI_optimization_goals: Aggressive Speed
  Tag_CPU_unaligned_access: None
  Tag_ABI_FP_16bit_format: IEEE 754

That's why linker is failed to linking the object files.

armv7-openwrt-linux-muslgnueabi :

root@router:~# readelf -A libunwind.o
Attribute Section: aeabi
File Attributes
  Tag_CPU_name: "7-A"
  Tag_CPU_arch: v7
  Tag_CPU_arch_profile: Application
  Tag_ARM_ISA_use: Yes
  Tag_THUMB_ISA_use: Thumb-2
  Tag_FP_arch: VFPv3-D16
  Tag_ABI_PCS_wchar_t: 4
  Tag_ABI_FP_denormal: Needed
  Tag_ABI_FP_exceptions: Needed
  Tag_ABI_FP_number_model: IEEE 754
  Tag_ABI_align_needed: 8-byte
  Tag_ABI_align_preserved: 8-byte, except leaf SP
  Tag_ABI_enum_size: int
  Tag_ABI_VFP_args: VFP registers
  Tag_ABI_optimization_goals: Aggressive Speed
  Tag_CPU_unaligned_access: v6

Working for arm_cortex-a15+neon-vfpv4_gcc-12.2.0_musl_eabi :

root@router:~# readelf -A libaio.so.1.0.2
Attribute Section: aeabi
File Attributes
  Tag_CPU_name: "7VE"
  Tag_CPU_arch: v7
  Tag_CPU_arch_profile: Application
  Tag_ARM_ISA_use: Yes
  Tag_THUMB_ISA_use: Thumb-2
  Tag_FP_arch: VFPv4
  Tag_Advanced_SIMD_arch: NEONv1 with Fused-MAC
  Tag_ABI_PCS_wchar_t: 4
  Tag_ABI_FP_number_model: IEEE 754
  Tag_ABI_align_needed: 8-byte
  Tag_ABI_enum_size: int
  Tag_ABI_VFP_args: VFP registers
  Tag_CPU_unaligned_access: v6
  Tag_MPextension_use: Allowed
  Tag_DIV_use: Allowed in v7-A with integer division extension
  Tag_Virtualization_use: TrustZone and Virtualization Extensions

I have no idea on how reproduce arm_cortex-a15+neon-vfpv4_gcc-12.2.0_musl_eabi binaries or libraries

@lu-zero
Copy link
Contributor Author

lu-zero commented Nov 12, 2022

I do not know enough how python-criptography is dealing with the cross compiling, I hope I can find time to look into the arch mapping.

@lu-zero
Copy link
Contributor Author

lu-zero commented Nov 18, 2022

I guess to make it work correctly we need to add all the mapping for the CFLAGS to RUSTFLAGS for the TARGET.

@njhsi
Copy link

njhsi commented Dec 14, 2022

Going further after adding :

TARGET_LDFLAGS += \
		-L$(PYTHON3_LIB_DIR)
export PYO3_CROSS_INCLUDE_DIR=$(STAGING_DIR)/usr/include
export PYO3_CROSS_LIB_DIR=$(STAGING_DIR)/usr/lib
export CARGO_BUILD_TARGET=arm-unknown-linux-musleabi

into the python-cryptography Makefile

But still error on linking :

nown-linux-musleabi/release/deps/cryptography_rust.unindent-748906483c1c119e.unindent.e81da170-cgu.7.rcgu.o.rcgu.o" "/home/username/works/OpenWrt/build_dir/target-arm_cortex-a15+neon-vfpv4_musl_eabi/pypi/cryptography-38.0.3/src/rust/target/arm-unknown-linux-musleabi/release/deps/cryptography_rust.unindent-748906483c1c119e.unindent.e81da170-cgu.8.rcgu.o.rcgu.o" "/home/username/works/OpenWrt/build_dir/target-arm_cortex-a15+neon-vfpv4_musl_eabi/pypi/cryptography-38.0.3/src/rust/target/arm-unknown-linux-musleabi/release/deps/cryptography_rust.unindent-748906483c1c119e.unindent.e81da170-cgu.9.rcgu.o.rcgu.o" "/home/username/works/OpenWrt/build_dir/target-arm_cortex-a15+neon-vfpv4_musl_eabi/pypi/cryptography-38.0.3/src/rust/target/arm-unknown-linux-musleabi/release/deps/cryptography_rust.unwind-3c091545acb97779.unwind.2e387473-cgu.0.rcgu.o.rcgu.o" "-Wl,--as-needed" "-L" "/home/username/works/OpenWrt/build_dir/target-arm_cortex-a15+neon-vfpv4_musl_eabi/pypi/cryptography-38.0.3/src/rust/target/arm-unknown-linux-musleabi/release/deps" "-L" "/home/username/works/OpenWrt/build_dir/target-arm_cortex-a15+neon-vfpv4_musl_eabi/pypi/cryptography-38.0.3/src/rust/target/release/deps" "-L" "/home/username/works/OpenWrt/staging_dir/host/lib64/rustlib/arm-unknown-linux-musleabi/lib" "-Wl,-Bstatic" "/home/username/works/OpenWrt/staging_dir/host/lib/rustlib/arm-unknown-linux-musleabi/lib/libcompiler_builtins-a637be64e8863187.rlib" "-Wl,-Bdynamic" "-lgcc_s" "-lc" "-Wl,--eh-frame-hdr" "-Wl,-znoexecstack" "-L" "/home/username/works/OpenWrt/staging_dir/host/lib64/rustlib/arm-unknown-linux-musleabi/lib" "-o" "/home/username/works/OpenWrt/build_dir/target-arm_cortex-a15+neon-vfpv4_musl_eabi/pypi/cryptography-38.0.3/src/rust/target/arm-unknown-linux-musleabi/release/deps/libcryptography_rust.so" "-Wl,--gc-sections" "-shared" "-Wl,-zrelro,-znow" "-Wl,-O1" "-nodefaultlibs"
  = note: /usr/bin/ld: /home/username/works/OpenWrt/build_dir/target-arm_cortex-a15+neon-vfpv4_musl_eabi/pypi/cryptography-38.0.3/src/rust/target/arm-unknown-linux-musleabi/release/deps/cryptography_rust.5d0d1978mqpjhaxk.rcgu.o: relocations in generic ELF (EM: 40)
          /usr/bin/ld: /home/username/works/OpenWrt/build_dir/target-arm_cortex-a15+neon-vfpv4_musl_eabi/pypi/cryptography-38.0.3/src/rust/target/arm-unknown-linux-musleabi/release/deps/cryptography_rust.5d0d1978mqpjhaxk.rcgu.o: relocations in generic ELF (EM: 40)
          /usr/bin/ld: /home/username/works/OpenWrt/build_dir/target-arm_cortex-a15+neon-vfpv4_musl_eabi/pypi/cryptography-38.0.3/src/rust/target/arm-unknown-linux-musleabi/release/deps/cryptography_rust.5d0d1978mqpjhaxk.rcgu.o: relocations in generic ELF (EM: 40)
          /usr/bin/ld: /home/username/works/OpenWrt/build_dir/target-arm_cortex-a15+neon-vfpv4_musl_eabi/pypi/cryptography-38.0.3/src/rust/target/arm-unknown-linux-musleabi/release/deps/cryptography_rust.5d0d1978mqpjhaxk.rcgu.o: relocations in generic ELF (EM: 40)
          /usr/bin/ld: /home/username/works/OpenWrt/build_dir/target-arm_cortex-a15+neon-vfpv4_musl_eabi/pypi/cryptography-38.0.3/src/rust/target/arm-unknown-linux-musleabi/release/deps/cryptography_rust.5d0d1978mqpjhaxk.rcgu.o: relocations in generic ELF (EM: 40)
          /usr/bin/ld: /home/username/works/OpenWrt/build_dir/target-arm_cortex-a15+neon-vfpv4_musl_eabi/pypi/cryptography-38.0.3/src/rust/target/arm-unknown-linux-musleabi/release/deps/cryptography_rust.5d0d1978mqpjhaxk.rcgu.o: relocations in generic ELF (EM: 40)
          /usr/bin/ld: /home/username/works/OpenWrt/build_dir/target-arm_cortex-a15+neon-vfpv4_musl_eabi/pypi/cryptography-38.0.3/src/rust/target/arm-unknown-linux-musleabi/release/deps/cryptography_rust.5d0d1978mqpjhaxk.rcgu.o: relocations in generic ELF (EM: 40)
          /usr/bin/ld: /home/username/works/OpenWrt/build_dir/target-arm_cortex-a15+neon-vfpv4_musl_eabi/pypi/cryptography-38.0.3/src/rust/target/arm-unknown-linux-musleabi/release/deps/cryptography_rust.5d0d1978mqpjhaxk.rcgu.o: relocations in generic ELF (EM: 40)
          /usr/bin/ld: /home/username/works/OpenWrt/build_dir/target-arm_cortex-a15+neon-vfpv4_musl_eabi/pypi/cryptography-38.0.3/src/rust/target/arm-unknown-linux-musleabi/release/deps/cryptography_rust.5d0d1978mqpjhaxk.rcgu.o: relocations in generic ELF (EM: 40)
          /usr/bin/ld: /home/username/works/OpenWrt/build_dir/target-arm_cortex-a15+neon-vfpv4_musl_eabi/pypi/cryptography-38.0.3/src/rust/target/arm-unknown-linux-musleabi/release/deps/cryptography_rust.5d0d1978mqpjhaxk.rcgu.o: relocations in generic ELF (EM: 40)
          /usr/bin/ld: /home/username/works/OpenWrt/build_dir/target-arm_cortex-a15+neon-vfpv4_musl_eabi/pypi/cryptography-38.0.3/src/rust/target/arm-unknown-linux-musleabi/release/deps/cryptography_rust.5d0d1978mqpjhaxk.rcgu.o: error adding symbols: file in wrong format
          collect2: error: ld returned 1 exit status


error: could not compile `cryptography-rust` due to previous error

Caused by:
  process didn't exit successfully: `rustc --crate-name cryptography_rust --edition=2018 src/lib.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --crate-type cdylib --emit=dep-info,link -C opt-level=3 -C lto=thin --crate-type cdylib -C overflow-checks=on --cfg 'feature="default"' --cfg 'feature="extension-module"' -C metadata=ad553caa0913c0d5 --out-dir /home/username/works/OpenWrt/build_dir/target-arm_cortex-a15+neon-vfpv4_musl_eabi/pypi/cryptography-38.0.3/src/rust/target/arm-unknown-linux-musleabi/release/deps --target arm-unknown-linux-musleabi -L dependency=/home/username/works/OpenWrt/build_dir/target-arm_cortex-a15+neon-vfpv4_musl_eabi/pypi/cryptography-38.0.3/src/rust/target/arm-unknown-linux-musleabi/release/deps -L dependency=/home/username/works/OpenWrt/build_dir/target-arm_cortex-a15+neon-vfpv4_musl_eabi/pypi/cryptography-38.0.3/src/rust/target/release/deps --extern asn1=/home/username/works/OpenWrt/build_dir/target-arm_cortex-a15+neon-vfpv4_musl_eabi/pypi/cryptography-38.0.3/src/rust/target/arm-unknown-linux-musleabi/release/deps/libasn1-d4315fe5564406fe.rlib --extern chrono=/home/username/works/OpenWrt/build_dir/target-arm_cortex-a15+neon-vfpv4_musl_eabi/pypi/cryptography-38.0.3/src/rust/target/arm-unknown-linux-musleabi/release/deps/libchrono-90a85769a2b0139c.rlib --extern once_cell=/home/username/works/OpenWrt/build_dir/target-arm_cortex-a15+neon-vfpv4_musl_eabi/pypi/cryptography-38.0.3/src/rust/target/arm-unknown-linux-musleabi/release/deps/libonce_cell-3057188e9b074a7a.rlib --extern ouroboros=/home/username/works/OpenWrt/build_dir/target-arm_cortex-a15+neon-vfpv4_musl_eabi/pypi/cryptography-38.0.3/src/rust/target/arm-unknown-linux-musleabi/release/deps/libouroboros-bf5a111e901f024b.rlib --extern pem=/home/username/works/OpenWrt/build_dir/target-arm_cortex-a15+neon-vfpv4_musl_eabi/pypi/cryptography-38.0.3/src/rust/target/arm-unknown-linux-musleabi/release/deps/libpem-81f502f57eda75c2.rlib --extern pyo3=/home/username/works/OpenWrt/build_dir/target-arm_cortex-a15+neon-vfpv4_musl_eabi/pypi/cryptography-38.0.3/src/rust/target/arm-unknown-linux-musleabi/release/deps/libpyo3-9e81ad91923da1fd.rlib -Ctarget-feature=-crt-static` (exit status: 1)

    =============================DEBUG ASSISTANCE=============================
    If you are seeing a compilation error please try the following steps to
    successfully install cryptography:
    1) Upgrade to the latest pip and try again. This will fix errors for most
       users. See: https://pip.pypa.io/en/stable/installing/#upgrading-pip
    2) Read https://cryptography.io/en/latest/installation/ for specific
       instructions for your platform.
    3) Check our frequently asked questions for more information:
       https://cryptography.io/en/latest/faq/
    4) Ensure you have a recent Rust toolchain installed:
       https://cryptography.io/en/latest/installation/#rust

    Python: 3.10.7
    platform: Linux-6.0.0-3-amd64-x86_64-with-glibc2.36
    pip: 22.2.2
    setuptools: 63.2.0
    setuptools_rust: 1.5.2
    rustc: 1.65.0
    =============================DEBUG ASSISTANCE=============================

error: `cargo rustc --lib --message-format=json-render-diagnostics --manifest-path src/rust/Cargo.toml --target arm-unknown-linux-musleabi --release -v --features 'pyo3/extension-module pyo3/abi3-py36' -- --crate-type cdylib` failed with code 101
make[2]: *** [Makefile:65: /home/username/works/OpenWrt/build_dir/target-arm_cortex-a15+neon-vfpv4_musl_eabi/pypi/cryptography-38.0.3/.built] Error 1
make[2]: Leaving directory '/home/username/works/OpenWrt/feeds/packages/lang/python/python-cryptography'
time: package/feeds/packages/python-cryptography/compile#72.23#20.37#22.66
    ERROR: package/feeds/packages/python-cryptography failed to build.
make[1]: *** [package/Makefile:116: package/feeds/packages/python-cryptography/compile] Error 1
make[1]: Leaving directory '/home/username/works/OpenWrt'
make: *** [/home/username/works/OpenWrt/include/toplevel.mk:231: package/python-cryptography/compile] Error 2

arm_cortex-a15+neon-vfpv4_gcc-12.2.0_musl_eabi cpuinfo :

root@router:~# cat /proc/cpuinfo
processor       : 0
model name      : ARMv7 Processor rev 0 (v7l)
BogoMIPS        : 12.50
Features        : half thumb fastmult vfp edsp neon vfpv3 tls vfpv4 idiva idivt vfpd32
CPU implementer : 0x51
CPU architecture: 7
CPU variant     : 0x2
CPU part        : 0x04d
CPU revision    : 0

processor       : 1
model name      : ARMv7 Processor rev 0 (v7l)
BogoMIPS        : 26.04
Features        : half thumb fastmult vfp edsp neon vfpv3 tls vfpv4 idiva idivt vfpd32
CPU implementer : 0x51
CPU architecture: 7
CPU variant     : 0x2
CPU part        : 0x04d
CPU revision    : 0

Hardware        : Generic DT based system
Revision        : 0000
Serial          : 0000000000000000

just tested arm-unknown-linux-musleabi RUSTC_TARGET_ARCH is resulting :

root@router:~# readelf -A cryptography_rust.5d0d1978mqpjhaxk.rcgu.o
Attribute Section: aeabi
File Attributes
  Tag_conformance: "2.09"
  Tag_CPU_arch: v6
  Tag_ARM_ISA_use: Yes
  Tag_THUMB_ISA_use: Thumb-1
  Tag_ABI_PCS_R9_use: V6
  Tag_ABI_PCS_RW_data: PC-relative
  Tag_ABI_PCS_RO_data: PC-relative
  Tag_ABI_PCS_GOT_use: GOT-indirect
  Tag_ABI_FP_denormal: Needed
  Tag_ABI_FP_exceptions: Unused
  Tag_ABI_FP_number_model: IEEE 754
  Tag_ABI_align_needed: 8-byte
  Tag_ABI_align_preserved: 8-byte, except leaf SP
  Tag_ABI_optimization_goals: Aggressive Speed
  Tag_CPU_unaligned_access: None
  Tag_ABI_FP_16bit_format: IEEE 754

That's why linker is failed to linking the object files.

armv7-openwrt-linux-muslgnueabi :

root@router:~# readelf -A libunwind.o
Attribute Section: aeabi
File Attributes
  Tag_CPU_name: "7-A"
  Tag_CPU_arch: v7
  Tag_CPU_arch_profile: Application
  Tag_ARM_ISA_use: Yes
  Tag_THUMB_ISA_use: Thumb-2
  Tag_FP_arch: VFPv3-D16
  Tag_ABI_PCS_wchar_t: 4
  Tag_ABI_FP_denormal: Needed
  Tag_ABI_FP_exceptions: Needed
  Tag_ABI_FP_number_model: IEEE 754
  Tag_ABI_align_needed: 8-byte
  Tag_ABI_align_preserved: 8-byte, except leaf SP
  Tag_ABI_enum_size: int
  Tag_ABI_VFP_args: VFP registers
  Tag_ABI_optimization_goals: Aggressive Speed
  Tag_CPU_unaligned_access: v6

Working for arm_cortex-a15+neon-vfpv4_gcc-12.2.0_musl_eabi :

root@router:~# readelf -A libaio.so.1.0.2
Attribute Section: aeabi
File Attributes
  Tag_CPU_name: "7VE"
  Tag_CPU_arch: v7
  Tag_CPU_arch_profile: Application
  Tag_ARM_ISA_use: Yes
  Tag_THUMB_ISA_use: Thumb-2
  Tag_FP_arch: VFPv4
  Tag_Advanced_SIMD_arch: NEONv1 with Fused-MAC
  Tag_ABI_PCS_wchar_t: 4
  Tag_ABI_FP_number_model: IEEE 754
  Tag_ABI_align_needed: 8-byte
  Tag_ABI_enum_size: int
  Tag_ABI_VFP_args: VFP registers
  Tag_CPU_unaligned_access: v6
  Tag_MPextension_use: Allowed
  Tag_DIV_use: Allowed in v7-A with integer division extension
  Tag_Virtualization_use: TrustZone and Virtualization Extensions

I have no idea on how reproduce arm_cortex-a15+neon-vfpv4_gcc-12.2.0_musl_eabi binaries or libraries

You're just on half way to bingo. just adjust the file "rust/patches/06-add_armv7.patch" to use actual features such as NEON and THUMB2. the original patch file lacks support of them, and mismatched to your toolchain.

If you're on a toolchain with "hard float", better adjust to it as well. for example:

--- /dev/null
+++ b/compiler/rustc_target/src/spec/armv7_openwrt_linux_muslgnueabi.rs
@@ -0,0 +1,26 @@
+use crate::spec::{Target, TargetOptions};
+
+// This target is for musl Linux on ARMv7 without thumb-mode, NEON or
+// hardfloat.
+
+pub fn target() -> Target {
+    // Most of these settings are copied from the armv7_unknown_linux_gnueabi
+    // target.
+    Target {
+        // It's important we use "gnueabi" and not "musleabi" here. LLVM uses it
+        // to determine the calling convention and float ABI, and it doesn't
+        // support the "musleabi" value.
+        llvm_target: "armv7-unknown-linux-gnueabihf".into(),
+        pointer_width: 32,
+        data_layout: "e-m:e-p:32:32-Fi8-i64:64-v128:64:128-a:0:32-n32-S64".into(),
+        arch: "arm".into(),
+
+        options: TargetOptions {
+            abi: "eabihf".into(),
+            features: "+v7,+vfp3,+neon".into(),
+            max_atomic_width: Some(64),
+            mcount: "\u{1}mcount".into(),
+            ..super::linux_musl_base::opts()
+        },
+    }
+}
--- a/compiler/rustc_target/src/spec/mod.rs
+++ b/compiler/rustc_target/src/spec/mod.rs
@@ -1017,6 +1017,7 @@
     ("mips-openwrt-linux-musl", mips_openwrt_linux_musl),
     ("mipsel-openwrt-linux-musl", mipsel_openwrt_linux_musl),
     ("aarch64-openwrt-linux-musl", aarch64_openwrt_linux_musl),
+    ("armv7-openwrt-linux-muslgnueabi", armv7_openwrt_linux_muslgnueabi),
 }

@njhsi
Copy link

njhsi commented Dec 14, 2022

" /usr/bin/ld: /home/username/works/OpenWrt/build_dir/target-arm_cortex-a15+neon-vfpv4_musl_eabi/pypi/cryptography-38.0.3/src/rust/target/arm-unknown-linux-musleabi/release/deps/cryptography_rust.5d0d1978mqpjhaxk.rcgu.o: relocations in generic ELF (EM: 40)
"

this was caused by wrong "cc" linker. check "which cc".
workaround is to re-link $(which cc) to your openwrt/toolchain/arm7..gcc.
I just compiled python-cryptograph 38 with rust succesfully.

@trippleflux
Copy link

trippleflux commented Dec 15, 2022

@njhsi

Thanks for the pointers, I was trying to compile mitmproxy as a test, have you test if python-cryptograph 38 is loaded correctly in the target device?

[EDIT]
I am new in rust world, is it possible to use rust with glibc?, specially x86_64.

@njhsi
Copy link

njhsi commented Dec 16, 2022

@trippleflux
I am not a bit knowing rust either, just need python-cryptograph38 for pyicloud on target, and it is loaded good and fit on target as pyicloud running there now.

I don't understand that of rust with glibc your mentioned, good luck anyway..

@lu-zero
Copy link
Contributor Author

lu-zero commented Dec 16, 2022

Thank you @njhsi I'll try to fold in your patch, or, even better, send a pr on top of my tree please.

@predators46
Copy link
Contributor

predators46 commented Dec 23, 2022

i tried to make ripgrep there is an error

Makefile:182: WARNING: skipping rust -- package has no install section                             
cd /home/open/ramips/build_dir/hostpkg/rustc-1.65.0-src/ && python3 x.py --config ./config.toml dist build-manifest cargo llvm-tools rustc rust-std rust-src                                          
Building rustbuild
Compiling bootstrap v0.0.0 (/home/open/ramips/build_dir/hostpkg/rustc-1.65.0-src/src/bootstrap) 
error: linking with `cc` failed: exit status: 1


= note: /usr/bin/ld: cannot find -llzma                                                                    
collect2: error: ld returned 1 exit status                                                                                                                                                                                                                                                     
error: could not compile `bootstrap` due to previous error```

Test
Ramips mt7620
openwrt-sdk-22.03.2

@lu-zero
Copy link
Contributor Author

lu-zero commented Dec 23, 2022

Looks like lzma is needed and your system does not have it.

@predators46
Copy link
Contributor

@lu-zero

I'm using debian 11 on azure. and I have installed lzma

@lu-zero
Copy link
Contributor Author

lu-zero commented Dec 23, 2022

If you try to manually run python3 x.py --config ./config.toml dist build-manifest cargo llvm-tools rustc rust-std rust-src does it present the same problem?

I'm halfway updating to rust 1.66, hopefully this weekend I'll have a new patch. I'm conflicted between try to remap the triples to the known ones and make the rustc as vanilla as possible or have the aliases mapped in a slightly less time consuming way.

@predators46
Copy link
Contributor

@lu-zero

cargo: /home/open/arch64/staging_dir/host/bin/../lib/libm.so.6: version `GLIBC_2.29' not found (required by cargo)                                                                                    
cargo: /home/open/arch64/staging_dir/host/bin/../lib/libc.so.6: version `GLIBC_2.29' not found (required by cargo)                                                                                    
make[2]: *** [Makefile:44: /home/open/arch64/build_dir/target-aarch64_cortex-a53_musl/ripgrep-13.0.0/.built] Error 1

@lu-zero
Copy link
Contributor Author

lu-zero commented Dec 24, 2022

It look like it is not linking the right libraries (host vs system), that's interesting since it works fine here.

@predators46
Copy link
Contributor

@predators46
Copy link
Contributor

@lu-zero

very strange
Screenshot_2022-12-25_133446

@predators46
Copy link
Contributor

predators46 commented Dec 25, 2022

@lu-zero
Error build ripgrep
i see cargo in /staging_dir/host/bin/

me/open/open/build_dir/target-mipsel_24kc_musl/ripgrep-13.0.0/ipkg-install --path "." --features 'pcre2'                                                                                              
*** stack smashing detected ***: <unknown> terminated                                              
bash: line 1:  4688 Aborted                 (core dumped) CARGO_HOME=/home/open/open/staging_dir/host cargo update                                                                                    
Makefile:40: recipe for target '/home/open/open/build_dir/target-mipsel_24kc_musl/ripgrep-13.0.0/.built' failed                                                                                       
make[2]: *** [/home/open/open/build_dir/target-mipsel_24kc_musl/ripgrep-13.0.0/.built] Error 13

@lu-zero
Copy link
Contributor Author

lu-zero commented Dec 25, 2022

@lu-zero Error build ripgrep i see cargo in /staging_dir/host/bin/

me/open/open/build_dir/target-mipsel_24kc_musl/ripgrep-13.0.0/ipkg-install --path "." --features 'pcre2'                                                                                              
*** stack smashing detected ***: <unknown> terminated                                              
bash: line 1:  4688 Aborted                 (core dumped) CARGO_HOME=/home/open/open/staging_dir/host cargo update                                                                                    
Makefile:40: recipe for target '/home/open/open/build_dir/target-mipsel_24kc_musl/ripgrep-13.0.0/.built' failed                                                                                       
make[2]: *** [/home/open/open/build_dir/target-mipsel_24kc_musl/ripgrep-13.0.0/.built] Error 13

This is bash crashing in a quite strange way.

lang/rust/Makefile Outdated Show resolved Hide resolved
lang/rust/maturin/Makefile Outdated Show resolved Hide resolved
lang/rust/maturin/Makefile Outdated Show resolved Hide resolved
lang/rust/maturin/Makefile Outdated Show resolved Hide resolved
lang/rust/maturin/Makefile Outdated Show resolved Hide resolved
lang/rust/rust-package.mk Outdated Show resolved Hide resolved
lang/rust/Makefile Outdated Show resolved Hide resolved
lang/rust/Makefile Outdated Show resolved Hide resolved
lang/rust/Makefile Outdated Show resolved Hide resolved
lang/rust/rust-host.mk Outdated Show resolved Hide resolved
lu-zero added 3 commits March 9, 2023 08:50
Based on work from Donald Hoskins <grommish@gmail.com>.

Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
Based on work from Donald Hoskins <grommish@gmail.com>.
Testcase for the rust language support.

Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
Build tool for rust-python packages.

Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
@lu-zero
Copy link
Contributor Author

lu-zero commented Mar 9, 2023

Thank you for all the fixes @1715173329 I hope the helper is what you suggested.

@1715173329
Copy link
Member

Thank you for all the fixes @1715173329 I hope the helper is what you suggested.

Thanks for your quick response!

Co-authored-by: Tianling Shen <cnsztl@gmail.com>
Signed-off-by: Luca Barbato <luca.barbato@gmail.com>
@lu-zero lu-zero force-pushed the master branch 2 times, most recently from f9fa94f to 221b59e Compare March 9, 2023 08:38
Copy link
Member

@1715173329 1715173329 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thank you for your great work!

lang/rust/maturin/Makefile Outdated Show resolved Hide resolved
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
@PolynomialDivision
Copy link
Member

ping @dangowrt

@dangowrt dangowrt merged commit 2d3e0da into openwrt:master Mar 10, 2023
@dangowrt
Copy link
Member

Now one run of the packages buildbot will take twice as long as before...

@sn99
Copy link

sn99 commented Mar 10, 2023

@dilawar This got merged

@stintel
Copy link
Member

stintel commented Mar 10, 2023

Thanks to everyone involved!

@PolynomialDivision
Copy link
Member

Now one run of the packages buildbot will take twice as long as before...

So we need more build Ressources ;)

@Akira25
Copy link
Contributor

Akira25 commented Mar 10, 2023

I would love to see this backported into the 22.03-branch too.

@dangowrt
Copy link
Member

Now that we got it in the development branch backporting it to openwrt-22.03 branch is a matter of cherry-picking the commits to this branch and making another PR for openwrt-22.03 branch.

@lu-zero
Copy link
Contributor Author

lu-zero commented Mar 10, 2023

I prepared the update for the current version of rust meanwhile. Hopefully it will survive the CI as the previous.

@PolynomialDivision
Copy link
Member

I prepared the update for the current version of rust meanwhile. Hopefully it will survive the CI as the previous.

At least the CI starts now automatically since you already contributed once to the repository. ;)

@lu-zero
Copy link
Contributor Author

lu-zero commented Mar 10, 2023

Yes, thank you a lot @PolynomialDivision for starting runs :) Now I'll need help figuring out why it fails though.

@jefferyto
Copy link
Member

Perhaps it would be better to let this be run-tested for a while by people who run master (who expect occasional instability), rather than immediately backport it to stable, run by people who expect stability?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.