Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement build for swift 5.7 #1

Closed
28 of 40 tasks
NikolayJuly opened this issue Dec 29, 2022 · 12 comments
Closed
28 of 40 tasks

Implement build for swift 5.7 #1

NikolayJuly opened this issue Dec 29, 2022 · 12 comments

Comments

@NikolayJuly
Copy link
Owner

NikolayJuly commented Dec 29, 2022

Plan:

@NikolayJuly
Copy link
Owner Author

Initial plan was to parse log from git, but it detects that there is no TTY and do not actually post status realtime. Found thread here. There are few other sggestions like empty, script or faketty. They didn't work or require additional work to install/compile. Or all together.

@NikolayJuly
Copy link
Owner Author

NikolayJuly commented Feb 5, 2023

StdLib

While building StdLib I encountered few issues.

Issue 1
When we build lib dispatch for std lib, we didn't pass 2 cmake cahe entries to it: ANDROID_PLATFORM and ANDROID_ABI. Because of it we got errors about API which presented only in API 21. Before this errors, while lib dispatch was configured there was warning that ANDROID_PLATFORM was not set and it was defaulted to 19.

To solve it, I introduced patches. Before building LibStd I patch repo with these values. Patch looks like this

diff --git a/cmake/modules/Libdispatch.cmake b/cmake/modules/Libdispatch.cmake
index 39b3e4bc531..b1b3bf4cc42 100644
--- a/cmake/modules/Libdispatch.cmake
+++ b/cmake/modules/Libdispatch.cmake
@@ -103,6 +103,7 @@ foreach(sdk ${DISPATCH_SDKS})
                           "-DCMAKE_ANDROID_NDK=${SWIFT_ANDROID_NDK_PATH}"
                           -DCMAKE_ANDROID_ARCH_ABI=${SWIFT_SDK_ANDROID_ARCH_${arch}_ABI}
                           -DCMAKE_ANDROID_API=${SWIFT_ANDROID_API_LEVEL}
+                          -DANDROID_PLATFORM=${ANDROID_PLATFORM}
                           -DBUILD_SHARED_LIBS=YES
                           -DENABLE_SWIFT=NO
                           -DBUILD_TESTING=NO
@@ -195,6 +196,8 @@ foreach(sdk ${DISPATCH_SDKS})
                             "-DCMAKE_ANDROID_NDK=${SWIFT_ANDROID_NDK_PATH}"
                             -DCMAKE_ANDROID_ARCH_ABI=${SWIFT_SDK_ANDROID_ARCH_${arch}_ABI}
                             -DCMAKE_ANDROID_API=${SWIFT_ANDROID_API_LEVEL}
+                            -DANDROID_PLATFORM=${ANDROID_PLATFORM}
+                            -DANDROID_ABI=${ANDROID_ABI}
                             -DBUILD_SHARED_LIBS=NO
                             -DENABLE_SWIFT=NO
                             -DBUILD_TESTING=NO

@NikolayJuly
Copy link
Owner Author

NikolayJuly commented Feb 5, 2023

StdLib Build Issue 2

Issue with linker

FAILED: lib/swift/android/aarch64/libswiftCore.so 
 : && /Users/<user-name>/Library/Android/sdk/ndk/25.1.8937393/toolchains/llvm/prebuilt/darwin-x86_64/bin/clang --target=aarch64-none-linux-android21 --sysroot=/Users/<user-name>/Library/Android/sdk/ndk/25.1.8937393/toolchains/llvm/prebuilt/darwin-x86_64/sysroot -fPIC -g -DANDROID -fdata-sections -ffunction-sections -funwind-tables -fstack-protector-strong -no-canonical-prefixes -D_FORTIFY_SOURCE=2 -Wformat -Werror=format-security  -fPIC -fno-semantic-interposition -Werror=date-time -Werror=unguarded-availability-new -Wall -Wextra -Wno-unused-parameter -Wwrite-strings -Wmissing-field-initializers -Wimplicit-fallthrough -Wcovered-switch-default -Wstring-conversion -Wmisleading-indentation -fdiagnostics-color -ffunction-sections -fdata-sections -O3 -DNDEBUG  -static-libstdc++ -Wl,--build-id=sha1 -Wl,--no-rosegment -Wl,--fatal-warnings -Wl,--gc-sections -Wl,--no-undefined -Qunused-arguments   -Wl,--color-diagnostics   -shared -Wl,-soname,libswiftCore.so -target aarch64-unknown-linux-android21 --sysroot=/Users/<user-name>/Library/Android/sdk/ndk/25.1.8937393/toolchains/llvm/prebuilt/darwin-x86_64/sysroot -B /Users/<user-name>/Library/Android/sdk/ndk/25.1.8937393/toolchains/llvm/prebuilt/darwin-x86_64/bin -lm -resource-dir=/Users/<user-name>/Library/Android/sdk/ndk/25.1.8937393/toolchains/llvm/prebuilt/darwin-x86_64/sysroot/../lib64/clang/14.0.6 -fuse-ld=lld -shared  -o lib/swift/android/aarch64/libswiftCore.so stdlib/public/core/ANDROID/aarch64/Swift.o stdlib/public/runtime/CMakeFiles/swiftRuntime-android-aarch64.dir/__/CompatibilityOverride/CompatibilityOverride.cpp.o stdlib/public/runtime/CMakeFiles/swiftRuntime-android-aarch64.dir/AnyHashableSupport.cpp.o stdlib/public/runtime/CMakeFiles/swiftRuntime-android-aarch64.dir/Array.cpp.o stdlib/public/runtime/CMakeFiles/swiftRuntime-android-aarch64.dir/AutoDiffSupport.cpp.o stdlib/public/runtime/CMakeFiles/swiftRuntime-android-aarch64.dir/Bincompat.cpp.o stdlib/public/runtime/CMakeFiles/swiftRuntime-android-aarch64.dir/Casting.cpp.o stdlib/public/runtime/CMakeFiles/swiftRuntime-android-aarch64.dir/CrashReporter.cpp.o stdlib/public/runtime/CMakeFiles/swiftRuntime-android-aarch64.dir/CygwinPort.cpp.o stdlib/public/runtime/CMakeFiles/swiftRuntime-android-aarch64.dir/Demangle.cpp.o stdlib/public/runtime/CMakeFiles/swiftRuntime-android-aarch64.dir/DynamicCast.cpp.o stdlib/public/runtime/CMakeFiles/swiftRuntime-android-aarch64.dir/Enum.cpp.o stdlib/public/runtime/CMakeFiles/swiftRuntime-android-aarch64.dir/EnvironmentVariables.cpp.o stdlib/public/runtime/CMakeFiles/swiftRuntime-android-aarch64.dir/ErrorObjectCommon.cpp.o stdlib/public/runtime/CMakeFiles/swiftRuntime-android-aarch64.dir/ErrorObjectNative.cpp.o stdlib/public/runtime/CMakeFiles/swiftRuntime-android-aarch64.dir/Errors.cpp.o stdlib/public/runtime/CMakeFiles/swiftRuntime-android-aarch64.dir/ErrorDefaultImpls.cpp.o stdlib/public/runtime/CMakeFiles/swiftRuntime-android-aarch64.dir/Exclusivity.cpp.o stdlib/public/runtime/CMakeFiles/swiftRuntime-android-aarch64.dir/ExistentialContainer.cpp.o stdlib/public/runtime/CMakeFiles/swiftRuntime-android-aarch64.dir/Float16Support.cpp.o stdlib/public/runtime/CMakeFiles/swiftRuntime-android-aarch64.dir/FoundationSupport.cpp.o stdlib/public/runtime/CMakeFiles/swiftRuntime-android-aarch64.dir/FunctionReplacement.cpp.o stdlib/public/runtime/CMakeFiles/swiftRuntime-android-aarch64.dir/Heap.cpp.o stdlib/public/runtime/CMakeFiles/swiftRuntime-android-aarch64.dir/HeapObject.cpp.o stdlib/public/runtime/CMakeFiles/swiftRuntime-android-aarch64.dir/ImageInspectionCommon.cpp.o stdlib/public/runtime/CMakeFiles/swiftRuntime-android-aarch64.dir/ImageInspectionMachO.cpp.o stdlib/public/runtime/CMakeFiles/swiftRuntime-android-aarch64.dir/ImageInspectionELF.cpp.o stdlib/public/runtime/CMakeFiles/swiftRuntime-android-aarch64.dir/ImageInspectionCOFF.cpp.o stdlib/public/runtime/CMakeFiles/swiftRuntime-android-aarch64.dir/ImageInspectionStatic.cpp.o stdlib/public/runtime/CMakeFiles/swiftRuntime-android-aarch64.dir/ImageInspectionWasm.cpp.o stdlib/public/runtime/CMakeFiles/swiftRuntime-android-aarch64.dir/KeyPaths.cpp.o stdlib/public/runtime/CMakeFiles/swiftRuntime-android-aarch64.dir/KnownMetadata.cpp.o stdlib/public/runtime/CMakeFiles/swiftRuntime-android-aarch64.dir/Metadata.cpp.o stdlib/public/runtime/CMakeFiles/swiftRuntime-android-aarch64.dir/MetadataLookup.cpp.o stdlib/public/runtime/CMakeFiles/swiftRuntime-android-aarch64.dir/MutexPThread.cpp.o stdlib/public/runtime/CMakeFiles/swiftRuntime-android-aarch64.dir/MutexWin32.cpp.o stdlib/public/runtime/CMakeFiles/swiftRuntime-android-aarch64.dir/Numeric.cpp.o stdlib/public/runtime/CMakeFiles/swiftRuntime-android-aarch64.dir/Once.cpp.o stdlib/public/runtime/CMakeFiles/swiftRuntime-android-aarch64.dir/Portability.cpp.o stdlib/public/runtime/CMakeFiles/swiftRuntime-android-aarch64.dir/ProtocolConformance.cpp.o stdlib/public/runtime/CMakeFiles/swiftRuntime-android-aarch64.dir/RefCount.cpp.o stdlib/public/runtime/CMakeFiles/swiftRuntime-android-aarch64.dir/ReflectionMirror.cpp.o stdlib/public/runtime/CMakeFiles/swiftRuntime-android-aarch64.dir/RuntimeInvocationsTracking.cpp.o stdlib/public/runtime/CMakeFiles/swiftRuntime-android-aarch64.dir/SwiftDtoa.cpp.o stdlib/public/runtime/CMakeFiles/swiftRuntime-android-aarch64.dir/SwiftTLSContext.cpp.o stdlib/public/runtime/CMakeFiles/swiftRuntime-android-aarch64.dir/AccessibleFunction.cpp.o stdlib/public/runtime/CMakeFiles/swiftRuntime-android-aarch64.dir/ErrorObject.mm.o stdlib/public/runtime/CMakeFiles/swiftRuntime-android-aarch64.dir/SwiftObject.mm.o stdlib/public/runtime/CMakeFiles/swiftRuntime-android-aarch64.dir/SwiftValue.mm.o stdlib/public/runtime/CMakeFiles/swiftRuntime-android-aarch64.dir/ReflectionMirrorObjC.mm.o stdlib/public/runtime/CMakeFiles/swiftRuntime-android-aarch64.dir/ObjCRuntimeGetImageNameFromClass.mm.o stdlib/public/LLVMSupport/CMakeFiles/swiftLLVMSupport-android-aarch64.dir/ErrorHandling.cpp.o stdlib/public/LLVMSupport/CMakeFiles/swiftLLVMSupport-android-aarch64.dir/Hashing.cpp.o stdlib/public/LLVMSupport/CMakeFiles/swiftLLVMSupport-android-aarch64.dir/MemAlloc.cpp.o stdlib/public/LLVMSupport/CMakeFiles/swiftLLVMSupport-android-aarch64.dir/SmallPtrSet.cpp.o stdlib/public/LLVMSupport/CMakeFiles/swiftLLVMSupport-android-aarch64.dir/SmallVector.cpp.o stdlib/public/LLVMSupport/CMakeFiles/swiftLLVMSupport-android-aarch64.dir/StringRef.cpp.o stdlib/public/CMakeFiles/swiftDemangling-android-aarch64.dir/__/__/lib/Demangling/Context.cpp.o stdlib/public/CMakeFiles/swiftDemangling-android-aarch64.dir/__/__/lib/Demangling/Demangler.cpp.o stdlib/public/CMakeFiles/swiftDemangling-android-aarch64.dir/__/__/lib/Demangling/ManglingUtils.cpp.o stdlib/public/CMakeFiles/swiftDemangling-android-aarch64.dir/__/__/lib/Demangling/NodePrinter.cpp.o stdlib/public/CMakeFiles/swiftDemangling-android-aarch64.dir/__/__/lib/Demangling/Punycode.cpp.o stdlib/public/CMakeFiles/swiftDemangling-android-aarch64.dir/__/__/lib/Demangling/Remangler.cpp.o stdlib/public/CMakeFiles/swiftDemangling-android-aarch64.dir/__/__/lib/Demangling/NodeDumper.cpp.o stdlib/public/CMakeFiles/swiftDemangling-android-aarch64.dir/__/__/lib/Demangling/Errors.cpp.o stdlib/public/stubs/CMakeFiles/swiftStdlibStubs-android-aarch64.dir/Assert.cpp.o stdlib/public/stubs/CMakeFiles/swiftStdlibStubs-android-aarch64.dir/GlobalObjects.cpp.o stdlib/public/stubs/CMakeFiles/swiftStdlibStubs-android-aarch64.dir/LibcShims.cpp.o stdlib/public/stubs/CMakeFiles/swiftStdlibStubs-android-aarch64.dir/Random.cpp.o stdlib/public/stubs/CMakeFiles/swiftStdlibStubs-android-aarch64.dir/Stubs.cpp.o stdlib/public/stubs/CMakeFiles/swiftStdlibStubs-android-aarch64.dir/ThreadLocalStorage.cpp.o stdlib/public/stubs/CMakeFiles/swiftStdlibStubs-android-aarch64.dir/MathStubs.cpp.o stdlib/public/stubs/CMakeFiles/swiftStdlibStubs-android-aarch64.dir/Unicode/UnicodeData.cpp.o stdlib/public/stubs/CMakeFiles/swiftStdlibStubs-android-aarch64.dir/Unicode/UnicodeGrapheme.cpp.o stdlib/public/stubs/CMakeFiles/swiftStdlibStubs-android-aarch64.dir/Unicode/UnicodeNormalization.cpp.o stdlib/public/stubs/CMakeFiles/swiftStdlibStubs-android-aarch64.dir/Unicode/UnicodeScalarProps.cpp.o stdlib/public/stubs/CMakeFiles/swiftStdlibStubs-android-aarch64.dir/Unicode/UnicodeWord.cpp.o stdlib/public/stubs/CMakeFiles/swiftStdlibStubs-android-aarch64.dir/Availability.mm.o stdlib/public/stubs/CMakeFiles/swiftStdlibStubs-android-aarch64.dir/FoundationHelpers.mm.o stdlib/public/stubs/CMakeFiles/swiftStdlibStubs-android-aarch64.dir/OptionalBridgingHelper.mm.o stdlib/public/stubs/CMakeFiles/swiftStdlibStubs-android-aarch64.dir/Reflection.mm.o stdlib/public/stubs/CMakeFiles/swiftStdlibStubs-android-aarch64.dir/SwiftNativeNSObject.mm.o stdlib/public/stubs/CMakeFiles/swiftStdlibStubs-android-aarch64.dir/SwiftNativeNSXXXBaseARC.m.o stdlib/public/stubs/CMakeFiles/swiftStdlibStubs-android-aarch64.dir/8/SwiftNativeNSXXXBase.mm.o stdlib/public/CommandLineSupport/CMakeFiles/swiftCommandLineSupport-android-aarch64.dir/CommandLine.cpp.o lib/swift/android/aarch64/swiftrt.o stdlib/public/core/CMakeFiles/swiftCore-android-aarch64.dir/__/__/linker-support/magic-symbols-for-install-name.c.o -L/Users/<user-name>/ws/SwiftAndroid_working/build/llvm-project/./lib   -L/Users/<user-name>/ws/SwiftAndroid_working/build/stdlib-aarch64/./lib/swift/android/aarch64   -L/Users/<user-name>/ws/SwiftAndroid_working/build/swift/bin/../lib/swift/android/aarch64   -L/Users/<user-name>/ws/SwiftAndroid_working/build/swift/bin/../lib/swift/android   -L/Users/<user-name>/Library/Android/sdk/ndk/25.1.8937393/toolchains/llvm/prebuilt/darwin-x86_64/sysroot/usr/lib/swift -ldl  -llog  /Users/<user-name>/Library/Android/sdk/ndk/25.1.8937393/toolchains/llvm/prebuilt/darwin-x86_64/sysroot/usr/lib/aarch64-linux-android/libc++abi.a  /Users/<user-name>/Library/Android/sdk/ndk/25.1.8937393/toolchains/llvm/prebuilt/darwin-x86_64/sysroot/usr/lib/aarch64-linux-android/libc++_shared.so  -latomic -lm && :
 ld.lld: error: undefined hidden symbol: __start_swift5_protocols
 >>> referenced by SwiftRT-ELF.cpp
 >>>               lib/swift/android/aarch64/swiftrt.o:(swift_image_constructor())
 >>> referenced by SwiftRT-ELF.cpp
 >>>               lib/swift/android/aarch64/swiftrt.o:(swift_image_constructor())
 >>> the encapsulation symbol needs to be retained under --gc-sections properly; consider -z nostart-stop-gc (see https://lld.llvm.org/ELF/start-stop-gc)
 ld.lld: error: undefined hidden symbol: __stop_swift5_protocols
 >>> referenced by SwiftRT-ELF.cpp
 >>>               lib/swift/android/aarch64/swiftrt.o:(swift_image_constructor())
 >>> referenced by SwiftRT-ELF.cpp
 >>>               lib/swift/android/aarch64/swiftrt.o:(swift_image_constructor())
 ld.lld: error: undefined hidden symbol: __start_swift5_protocol_conformances
 >>> referenced by SwiftRT-ELF.cpp
 >>>               lib/swift/android/aarch64/swiftrt.o:(swift_image_constructor())
 >>> referenced by SwiftRT-ELF.cpp
 >>>               lib/swift/android/aarch64/swiftrt.o:(swift_image_constructor())
 >>> the encapsulation symbol needs to be retained under --gc-sections properly; consider -z nostart-stop-gc (see https://lld.llvm.org/ELF/start-stop-gc)
 ld.lld: error: undefined hidden symbol: __stop_swift5_protocol_conformances
 >>> referenced by SwiftRT-ELF.cpp
 >>>               lib/swift/android/aarch64/swiftrt.o:(swift_image_constructor())
 >>> referenced by SwiftRT-ELF.cpp
 >>>               lib/swift/android/aarch64/swiftrt.o:(swift_image_constructor())
 ld.lld: error: undefined hidden symbol: __start_swift5_type_metadata
 >>> referenced by SwiftRT-ELF.cpp
 >>>               lib/swift/android/aarch64/swiftrt.o:(swift_image_constructor())
 >>> referenced by SwiftRT-ELF.cpp
 >>>               lib/swift/android/aarch64/swiftrt.o:(swift_image_constructor())
 >>> the encapsulation symbol needs to be retained under --gc-sections properly; consider -z nostart-stop-gc (see https://lld.llvm.org/ELF/start-stop-gc)
 ld.lld: error: undefined hidden symbol: __stop_swift5_type_metadata
 >>> referenced by SwiftRT-ELF.cpp
 >>>               lib/swift/android/aarch64/swiftrt.o:(swift_image_constructor())
 >>> referenced by SwiftRT-ELF.cpp
 >>>               lib/swift/android/aarch64/swiftrt.o:(swift_image_constructor())
 ld.lld: error: undefined hidden symbol: __start_swift5_assocty
 >>> referenced by SwiftRT-ELF.cpp
 >>>               lib/swift/android/aarch64/swiftrt.o:(swift_image_constructor())
 >>> referenced by SwiftRT-ELF.cpp
 >>>               lib/swift/android/aarch64/swiftrt.o:(swift_image_constructor())
 >>> the encapsulation symbol needs to be retained under --gc-sections properly; consider -z nostart-stop-gc (see https://lld.llvm.org/ELF/start-stop-gc)
 ld.lld: error: undefined hidden symbol: __stop_swift5_assocty
 >>> referenced by SwiftRT-ELF.cpp
 >>>               lib/swift/android/aarch64/swiftrt.o:(swift_image_constructor())
 >>> referenced by SwiftRT-ELF.cpp
 >>>               lib/swift/android/aarch64/swiftrt.o:(swift_image_constructor())
 ld.lld: error: undefined hidden symbol: __start_swift5_replace
 >>> referenced by SwiftRT-ELF.cpp
 >>>               lib/swift/android/aarch64/swiftrt.o:(swift_image_constructor())
 >>> referenced by SwiftRT-ELF.cpp
 >>>               lib/swift/android/aarch64/swiftrt.o:(swift_image_constructor())
 >>> the encapsulation symbol needs to be retained under --gc-sections properly; consider -z nostart-stop-gc (see https://lld.llvm.org/ELF/start-stop-gc)
 ld.lld: error: undefined hidden symbol: __stop_swift5_replace
 >>> referenced by SwiftRT-ELF.cpp
 >>>               lib/swift/android/aarch64/swiftrt.o:(swift_image_constructor())
 >>> referenced by SwiftRT-ELF.cpp
 >>>               lib/swift/android/aarch64/swiftrt.o:(swift_image_constructor())
 ld.lld: error: undefined hidden symbol: __start_swift5_replac2
 >>> referenced by SwiftRT-ELF.cpp
 >>>               lib/swift/android/aarch64/swiftrt.o:(swift_image_constructor())
 >>> referenced by SwiftRT-ELF.cpp
 >>>               lib/swift/android/aarch64/swiftrt.o:(swift_image_constructor())
 >>> the encapsulation symbol needs to be retained under --gc-sections properly; consider -z nostart-stop-gc (see https://lld.llvm.org/ELF/start-stop-gc)
 ld.lld: error: undefined hidden symbol: __stop_swift5_replac2
 >>> referenced by SwiftRT-ELF.cpp
 >>>               lib/swift/android/aarch64/swiftrt.o:(swift_image_constructor())
 >>> referenced by SwiftRT-ELF.cpp
 >>>               lib/swift/android/aarch64/swiftrt.o:(swift_image_constructor())
 ld.lld: error: undefined hidden symbol: __start_swift5_builtin
 >>> referenced by SwiftRT-ELF.cpp
 >>>               lib/swift/android/aarch64/swiftrt.o:(swift_image_constructor())
 >>> referenced by SwiftRT-ELF.cpp
 >>>               lib/swift/android/aarch64/swiftrt.o:(swift_image_constructor())
 >>> the encapsulation symbol needs to be retained under --gc-sections properly; consider -z nostart-stop-gc (see https://lld.llvm.org/ELF/start-stop-gc)
 ld.lld: error: undefined hidden symbol: __stop_swift5_builtin
 >>> referenced by SwiftRT-ELF.cpp
 >>>               lib/swift/android/aarch64/swiftrt.o:(swift_image_constructor())
 >>> referenced by SwiftRT-ELF.cpp
 >>>               lib/swift/android/aarch64/swiftrt.o:(swift_image_constructor())
 ld.lld: error: undefined hidden symbol: __start_swift5_mpenum
 >>> referenced by SwiftRT-ELF.cpp
 >>>               lib/swift/android/aarch64/swiftrt.o:(swift_image_constructor())
 >>> referenced by SwiftRT-ELF.cpp
 >>>               lib/swift/android/aarch64/swiftrt.o:(swift_image_constructor())
 >>> the encapsulation symbol needs to be retained under --gc-sections properly; consider -z nostart-stop-gc (see https://lld.llvm.org/ELF/start-stop-gc)
 ld.lld: error: undefined hidden symbol: __stop_swift5_mpenum
 >>> referenced by SwiftRT-ELF.cpp
 >>>               lib/swift/android/aarch64/swiftrt.o:(swift_image_constructor())
 >>> referenced by SwiftRT-ELF.cpp
 >>>               lib/swift/android/aarch64/swiftrt.o:(swift_image_constructor())
 ld.lld: error: undefined hidden symbol: __start_swift5_accessible_functions
 >>> referenced by SwiftRT-ELF.cpp
 >>>               lib/swift/android/aarch64/swiftrt.o:(swift_image_constructor())
 >>> referenced by SwiftRT-ELF.cpp
 >>>               lib/swift/android/aarch64/swiftrt.o:(swift_image_constructor())
 >>> the encapsulation symbol needs to be retained under --gc-sections properly; consider -z nostart-stop-gc (see https://lld.llvm.org/ELF/start-stop-gc)
 ld.lld: error: undefined hidden symbol: __stop_swift5_accessible_functions
 >>> referenced by SwiftRT-ELF.cpp
 >>>               lib/swift/android/aarch64/swiftrt.o:(swift_image_constructor())
 >>> referenced by SwiftRT-ELF.cpp
 >>>               lib/swift/android/aarch64/swiftrt.o:(swift_image_constructor())
 clang: error: linker command failed with exit code 1 (use -v to see invocation)
 [194/224] Building C object stdlib/private/SwiftPrivateThreadExtras/CMakeFiles/swiftSwiftPrivateThreadExtras-android-aarch64.dir/__/__/linker-support/magic-symbols-for-install-name.c.o
 [195/224] Building CXX object stdlib/public/Concurrency/CMakeFiles/swift_Concurrency-android-aarch64.dir/TaskGroup.cpp.o
 [196/224] Building CXX object stdlib/public/Concurrency/CMakeFiles/swift_Concurrency-android-aarch64.dir/ThreadSanitizer.cpp.o
 ninja: build stopped: subcommand failed.

As I undestood, it is connected to default usage of -z start-stop-gc in lld by default. Solution taken form here

Patch loks like this

diff --git a/stdlib/cmake/modules/AddSwiftStdlib.cmake b/stdlib/cmake/modules/AddSwiftStdlib.cmake
index b41261feb2b..a33ccc3bfac 100644
--- a/stdlib/cmake/modules/AddSwiftStdlib.cmake
+++ b/stdlib/cmake/modules/AddSwiftStdlib.cmake
@@ -507,6 +507,9 @@ function(_add_target_variant_link_flags)
     endif()
   endif()
 
+  list(APPEND result "-Wl,-z")
+  list(APPEND result "-Wl,nostart-stop-gc")
+
   get_maccatalyst_build_flavor(maccatalyst_build_flavor
     "${LFLAGS_SDK}" "${LFLAGS_MACCATALYST_BUILD_FLAVOR}")

@NikolayJuly
Copy link
Owner Author

libDispatch

Issue 1

FAILED: libswiftDispatch.so src/swift/CMakeFiles/swiftDispatch.dir/Block.swift.o src/swift/CMakeFiles/swiftDispatch.dir/Data.swift.o src/swift/CMakeFiles/swiftDispatch.dir/Dispatch.swift.o src/swift/CMakeFiles/swiftDispatch.dir/IO.swift.o src/swift/CMakeFiles/swiftDispatch.dir/Private.swift.o src/swift/CMakeFiles/swiftDispatch.dir/Queue.swift.o src/swift/CMakeFiles/swiftDispatch.dir/Source.swift.o src/swift/CMakeFiles/swiftDispatch.dir/Time.swift.o src/swift/CMakeFiles/swiftDispatch.dir/Wrapper.swift.o src/swift/swift/Dispatch.swiftmodule 

/Users/${USER}/ws/SwiftAndroid_working/build/swift/bin/swiftc -v -target aarch64-unknown-linux-android -output-file-map src/swift/CMakeFiles/swiftDispatch.dir/Release/output-file-map.json -incremental -j 10 -emit-library -o libswiftDispatch.so -module-name Dispatch -module-link-name swiftDispatch -emit-module -emit-module-path src/swift/swift/Dispatch.swiftmodule -emit-dependencies -DswiftDispatch_EXPORTS -resource-dir /Users/${USER}/ws/SwiftAndroid_working/build/stdlib-aarch64/lib/swift -Xcc --sysroot=/Users/${USER}/Library/Android/sdk/ndk/25.1.8937393/toolchains/llvm/prebuilt/darwin-x86_64/sysroot -Xclang-linker --sysroot=/Users/${USER}/Library/Android/sdk/ndk/25.1.8937393/toolchains/llvm/prebuilt/darwin-x86_64/sysroot/usr/lib/aarch64-linux-android/21 -Xclang-linker --gcc-toolchain=/Users/${USER}/Library/Android/sdk/ndk/25.1.8937393/toolchains/llvm/prebuilt/darwin-x86_64 -O -Xcc -fblocks -Xcc -fmodule-map-file=/Users/${USER}/ws/SwiftAndroid_working/swift-corelibs-libdispatch/dispatch/module.modulemap -Xcc -I/Users/${USER}/ws/SwiftAndroid_working/swift-corelibs-libdispatch -Xcc -I/Users/${USER}/ws/SwiftAndroid_working/swift-corelibs-libdispatch/src/swift/shims -I /Users/${USER}/ws/SwiftAndroid_working/swift-corelibs-libdispatch/src/BlocksRuntime -I /Users/${USER}/ws/SwiftAndroid_working/build/libDispatch-aarch64 -I /Users/${USER}/ws/SwiftAndroid_working/swift-corelibs-libdispatch -I /Users/${USER}/ws/SwiftAndroid_working/swift-corelibs-libdispatch/src -I /Users/${USER}/ws/SwiftAndroid_working/build/libDispatch-aarch64/src /Users/${USER}/ws/SwiftAndroid_working/swift-corelibs-libdispatch/src/swift/Block.swift /Users/${USER}/ws/SwiftAndroid_working/swift-corelibs-libdispatch/src/swift/Data.swift /Users/${USER}/ws/SwiftAndroid_working/swift-corelibs-libdispatch/src/swift/Dispatch.swift /Users/${USER}/ws/SwiftAndroid_working/swift-corelibs-libdispatch/src/swift/IO.swift /Users/${USER}/ws/SwiftAndroid_working/swift-corelibs-libdispatch/src/swift/Private.swift /Users/${USER}/ws/SwiftAndroid_working/swift-corelibs-libdispatch/src/swift/Queue.swift /Users/${USER}/ws/SwiftAndroid_working/swift-corelibs-libdispatch/src/swift/Source.swift /Users/${USER}/ws/SwiftAndroid_working/swift-corelibs-libdispatch/src/swift/Time.swift /Users/${USER}/ws/SwiftAndroid_working/swift-corelibs-libdispatch/src/swift/Wrapper.swift -no-toolchain-stdlib-rpath -Xlinker -soname -Xlinker libswiftDispatch.so  -L /Users/${USER}/ws/SwiftAndroid_working/build/libDispatch-aarch64/src/swift  -L /Users/${USER}/ws/SwiftAndroid_working/build/libDispatch-aarch64  -L /Users/${USER}/ws/SwiftAndroid_working/build/libDispatch-aarch64   -L /Users/${USER}/Library/Android/sdk/ndk/25.1.8937393/toolchains/llvm/prebuilt/darwin-x86_64/lib64/clang/14.0.6/lib/linux/aarch64   -L /Users/${USER}/Library/Android/sdk/ndk/25.1.8937393/toolchains/llvm/prebuilt/darwin-x86_64/sysroot/usr/lib/aarch64-linux-android/21   -L /Users/${USER}/Library/Android/sdk/ndk/25.1.8937393/toolchains/llvm/prebuilt/darwin-x86_64/sysroot/usr/lib/aarch64-linux-android   -L /Users/${USER}/Library/Android/sdk/ndk/25.1.8937393/toolchains/llvm/prebuilt/darwin-x86_64/sysroot/usr/lib -Xlinker -rpath -Xlinker "\$ORIGIN"  src/swift/libDispatchStubs.a  libdispatch.so  libBlocksRuntime.so  -lc++  -lm  -l:libunwind.a  -ldl  -lc  -l:libunwind.a  -ldl && :

clang: error: invalid linker name in argument '-fuse-ld=lld'
<unknown>:0: error: link command failed with exit code 1 (use -v to see invocation)
ninja: build stopped: subcommand failed.

I added/uncommented verbose options for swift

            "-Xclang-linker -v",
            "-v",

And it appears that swiftc uses pre-installed apple's clang and linker as result.
I can see it in logs

/usr/bin/clang -shared -fuse-ld=lld /Users/${USER}/ws/SwiftAndroid_working/build/stdlib-aarch64/lib/swift/android/aarch64/swiftrt.o src/swift/CMakeFiles/swiftDispatch.dir/Block.swift.o src/swift/CMakeFiles/swiftDispatch.dir/Data.swift.o src/swift/CMakeFiles/swiftDispatch.dir/Dispatch.swift.o src/swift/CMakeFiles/swiftDispatch.dir/IO.swift.o src/swift/CMakeFiles/swiftDispatch.dir/Private.swift.o src/swift/CMakeFiles/swiftDispatch.dir/Queue.swift.o src/swift/CMakeFiles/swiftDispatch.dir/Source.swift.o src/swift/CMakeFiles/swiftDispatch.dir/Time.swift.o src/swift/CMakeFiles/swiftDispatch.dir/Wrapper.swift.o src/swift/libDispatchStubs.a libdispatch.so libBlocksRuntime.so @/var/folders/kn/stbgzggs3gb3ykqzz43bb_rw0000gn/T/libDispatchStubs-c67b27.autolink -L /Users/${USER}/ws/SwiftAndroid_working/build/stdlib-aarch64/lib/swift/android -lswiftCore --target=aarch64-unknown-linux-android -v -L /Users/${USER}/ws/SwiftAndroid_working/build/libDispatch-aarch64/src/swift -L /Users/${USER}/ws/SwiftAndroid_working/build/libDispatch-aarch64 -L /Users/${USER}/ws/SwiftAndroid_working/build/libDispatch-aarch64 -L /Users/${USER}/Library/Android/sdk/ndk/25.1.8937393/toolchains/llvm/prebuilt/darwin-x86_64/lib64/clang/14.0.6/lib/linux/aarch64 -L /Users/${USER}/Library/Android/sdk/ndk/25.1.8937393/toolchains/llvm/prebuilt/darwin-x86_64/sysroot/usr/lib/aarch64-linux-android/21 -L /Users/${USER}/Library/Android/sdk/ndk/25.1.8937393/toolchains/llvm/prebuilt/darwin-x86_64/sysroot/usr/lib/aarch64-linux-android -L /Users/${USER}/Library/Android/sdk/ndk/25.1.8937393/toolchains/llvm/prebuilt/darwin-x86_64/sysroot/usr/lib -lc++ -lm -l:libunwind.a -ldl -lc -l:libunwind.a -ldl -Xlinker -soname -Xlinker libswiftDispatch.so -Xlinker -rpath -Xlinker "\$ORIGIN" --sysroot=/Users/${USER}/Library/Android/sdk/ndk/25.1.8937393/toolchains/llvm/prebuilt/darwin-x86_64/sysroot/usr/lib/aarch64-linux-android/21 --gcc-toolchain=/Users/${USER}/Library/Android/sdk/ndk/25.1.8937393/toolchains/llvm/prebuilt/darwin-x86_64 -v -o libswiftDispatch.so
Target: aarch64-unknown-linux-android
Thread model: posix
InstalledDir: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin
clang: error: invalid linker name in argument '-fuse-ld=lld'
<unknown>:0: error: link command failed with exit code 1 (use -v to see invocation)
ninja: build stopped: subcommand failed.

Solution
Swift has -tools-directory parametr, I set it to NDK path + toolchains/llvm/prebuilt/darwin-x86_64/bin, where clang and lld are located

@NikolayJuly
Copy link
Owner Author

NikolayJuly commented Feb 11, 2023

Issue 2
After switching to NDKs clang and linker, I've got this error

ld.lld: error: cannot open crtbegin_so.o: No such file or directory
ld.lld: error: cannot open crtend_so.o: No such file or directory
clang-14: error: linker command failed with exit code 1 (use -v to see invocation)
<unknown>:0: error: link command failed with exit code 1 (use -v to see invocation)

I can see that files are in NDK folders somehwere, but they are not in the same folder as lld and don't have absolute path, like other object files

Here is final command to ld.lld
"/Users/${USER}/Library/Android/sdk/ndk/25.1.8937393/toolchains/llvm/prebuilt/darwin-x86_64/bin/ld.lld" \
--sysroot=/Users/${USER}/Library/Android/sdk/ndk/25.1.8937393/toolchains/llvm/prebuilt/darwin-x86_64/sysroot/usr/lib/aarch64-linux-android/21 \
-EL \
--fix-cortex-a53-843419 \
--warn-shared-textrel \
-z \
now \
-z \
relro \
-z \
max-page-size=4096 \
--hash-style=both \
--enable-new-dtags \
--eh-frame-hdr \
-m \
aarch64linux \
-shared \
-o \
/Users/${USER}/ws/SwiftAndroid_working/build/libDispatch-aarch64/libswiftDispatch.so \
crtbegin_so.o \
-L/Users/${USER}/ws/SwiftAndroid_working/build/stdlib-aarch64/lib/swift/android \
-L/Users/${USER}/ws/SwiftAndroid_working/build/libDispatch-aarch64/src/swift \
-L/Users/${USER}/ws/SwiftAndroid_working/build/libDispatch-aarch64 \
-L/Users/${USER}/ws/SwiftAndroid_working/build/libDispatch-aarch64 \
-L/Users/${USER}/Library/Android/sdk/ndk/25.1.8937393/toolchains/llvm/prebuilt/darwin-x86_64/lib64/clang/14.0.6/lib/linux/aarch64 \
-L/Users/${USER}/Library/Android/sdk/ndk/25.1.8937393/toolchains/llvm/prebuilt/darwin-x86_64/sysroot/usr/lib/aarch64-linux-android/21 \
-L/Users/${USER}/Library/Android/sdk/ndk/25.1.8937393/toolchains/llvm/prebuilt/darwin-x86_64/sysroot/usr/lib/aarch64-linux-android \
-L/Users/${USER}/Library/Android/sdk/ndk/25.1.8937393/toolchains/llvm/prebuilt/darwin-x86_64/sysroot/usr/lib \
-L/Users/${USER}/Library/Android/sdk/ndk/25.1.8937393/toolchains/llvm/prebuilt/darwin-x86_64/lib64/clang/14.0.6/lib/linux/aarch64 \
/Users/${USER}/ws/SwiftAndroid_working/build/stdlib-aarch64/lib/swift/android/aarch64/swiftrt.o \
/Users/${USER}/ws/SwiftAndroid_working/build/libDispatch-aarch64/src/swift/CMakeFiles/swiftDispatch.dir/Block.swift.o \
/Users/${USER}/ws/SwiftAndroid_working/build/libDispatch-aarch64/src/swift/CMakeFiles/swiftDispatch.dir/Data.swift.o \
/Users/${USER}/ws/SwiftAndroid_working/build/libDispatch-aarch64/src/swift/CMakeFiles/swiftDispatch.dir/Dispatch.swift.o \
/Users/${USER}/ws/SwiftAndroid_working/build/libDispatch-aarch64/src/swift/CMakeFiles/swiftDispatch.dir/IO.swift.o \
/Users/${USER}/ws/SwiftAndroid_working/build/libDispatch-aarch64/src/swift/CMakeFiles/swiftDispatch.dir/Private.swift.o \
/Users/${USER}/ws/SwiftAndroid_working/build/libDispatch-aarch64/src/swift/CMakeFiles/swiftDispatch.dir/Queue.swift.o \
/Users/${USER}/ws/SwiftAndroid_working/build/libDispatch-aarch64/src/swift/CMakeFiles/swiftDispatch.dir/Source.swift.o \
/Users/${USER}/ws/SwiftAndroid_working/build/libDispatch-aarch64/src/swift/CMakeFiles/swiftDispatch.dir/Time.swift.o \
/Users/${USER}/ws/SwiftAndroid_working/build/libDispatch-aarch64/src/swift/CMakeFiles/swiftDispatch.dir/Wrapper.swift.o \
/Users/${USER}/ws/SwiftAndroid_working/build/libDispatch-aarch64/src/swift/libDispatchStubs.a \
/Users/${USER}/ws/SwiftAndroid_working/build/libDispatch-aarch64/libdispatch.so \
/Users/${USER}/ws/SwiftAndroid_working/build/libDispatch-aarch64/libBlocksRuntime.so \
-ldispatch \
-ldispatch \
-ldispatch \
-ldispatch \
-ldispatch \
-ldispatch \
-ldispatch \
-ldispatch \
-ldispatch \
-lswift_Concurrency \
-lswiftCore \
-lswiftCore \
-lc++ \
-lm \
-l:libunwind.a \
-ldl \
-lc \
-l:libunwind.a \
-ldl \
-soname \
libswiftDispatch.so \
-rpath \
"\$ORIGIN" \
/Users/${USER}/Library/Android/sdk/ndk/25.1.8937393/toolchains/llvm/prebuilt/darwin-x86_64/lib64/clang/14.0.6/lib/linux/libclang_rt.builtins-aarch64-android.a \
-l:libunwind.a \
-ldl \
-lc \
/Users/${USER}/Library/Android/sdk/ndk/25.1.8937393/toolchains/llvm/prebuilt/darwin-x86_64/lib64/clang/14.0.6/lib/linux/libclang_rt.builtins-aarch64-android.a \
-l:libunwind.a \
-ldl \
crtend_so.o

I can see that lld was pointed to correct folder to search for those files - /Users/${USER}/Library/Android/sdk/ndk/25.1.8937393/toolchains/llvm/prebuilt/darwin-x86_64/sysroot/usr/lib/aarch64-linux-android/21

Solution

I can see that these files passed by clang and I didn't find the way to force clang to pass proper pathes.
But I found this issue on stackverflow. They proposed -nostartfiles and I've seen same option when build stdlib. So I added this line

"-Xclang-linker -nostartfiles",

If issue repeats in future, might consider other work around proposed in the same issue on StackOverflow, copy files, possible next to lld or in working folder, so it can find them. But this is dirty hack

@NikolayJuly
Copy link
Owner Author

NikolayJuly commented Mar 12, 2023

libFoundation

Issue 1

It appears libFoundation require ICU. I heard news that Apple dropped ICU requirement for swift, and assumed that it also included all libs. Apperently not. So I need to implement checkout and build for ICU, which is not ninja, it uses configure + make

@NikolayJuly
Copy link
Owner Author

libFoundation Issue 2

After building openssl and libxml2, I was able to pass configure command in foundation. And I found other error

ld.lld: error: undefined symbol: pthread_atfork
>>> referenced by libc_init_common.cpp:132 (bionic/libc/bionic/libc_init_common.cpp:132)
>>>               libc_init_common.o:(__libc_init_fork_handler()) in archive /Users/nikolaydzhulay/Library/Android/sdk/ndk/25.1.8937393/toolchains/llvm/prebuilt/darwin-x86_64/sysroot/usr/lib/aarch64-linux-android/libc.a
>>> referenced by android_je_iterate.c:91 (external/jemalloc_new/src/android_je_iterate.c:91)
>>>               jemalloc.o:(je_malloc_disable_init) in archive /Users/nikolaydzhulay/Library/Android/sdk/ndk/25.1.8937393/toolchains/llvm/prebuilt/darwin-x86_64/sysroot/usr/lib/aarch64-linux-android/libc.a
>>> referenced by jemalloc.c:1367 (external/jemalloc_new/src/jemalloc.c:1367)
>>>               jemalloc.o:(malloc_init_hard) in archive /Users/nikolaydzhulay/Library/Android/sdk/ndk/25.1.8937393/toolchains/llvm/prebuilt/darwin-x86_64/sysroot/usr/lib/aarch64-linux-android/libc.a
>>> referenced 1 more times

ld.lld: error: undefined symbol: atexit
>>> referenced by jemalloc.c:1291 (external/jemalloc_new/src/jemalloc.c:1291)
>>>               jemalloc.o:(malloc_init_hard_a0_locked) in archive /Users/nikolaydzhulay/Library/Android/sdk/ndk/25.1.8937393/toolchains/llvm/prebuilt/darwin-x86_64/sysroot/usr/lib/aarch64-linux-android/libc.a
clang-14: error: linker command failed with exit code 1 (use -v to see invocation)
<unknown>:0: error: link command failed with exit code 1 (use -v to see invocation)

Error occurred when we link plutil, which is added to build by default.

As I understand here is opposite problem, of what I had before. I need to link with crtbegin_so.o

We building executable, and no easy way to disable it. Only way is apply patch, which I would prefer to avoid.

@NikolayJuly
Copy link
Owner Author

NikolayJuly commented Jul 29, 2023

While compiled shared library I got familiar error about missing crtbegin_so.o and crtend_so.o. This time I used another solution: -B parameter for clang. Which provide search path for binaries.
Here is example:

    "-Xclang-linker", "-B",
    "-Xclang-linker", "<path>",

Need to split in 2, so we path -B and <path>

<path> - is fodler where crtbegin_so.o located

@NikolayJuly
Copy link
Owner Author

NikolayJuly commented Aug 13, 2023

After compiling executable binary for test deployment on emulator I run into few issues.

Need to copy all dependencies from toolchain
I need to copy many .so files, command looks like this

/Users/$USER/Library/Android/sdk/platform-tools/adb -s emulator-5566 push <path_to_toolchain>/usr/lib/swift/android/aarch64/libFoundation.so /data/bin/libFoundation.so

Repeat for every missded .so file

Need to copy some libs from NDK

/Users/$USER/Library/Android/sdk/platform-tools/adb -s emulator-5566 push /Users/$USER/Library/Android/sdk/ndk/25.1.8937393/toolchains/llvm/prebuilt/darwin-x86_64/sysroot/usr/lib/aarch64-linux-android/libc++_shared.so /data/bin/libc++_shared.so

Additional LD path
While running app on emulator, ld can't find all needed .so files even tho they are in the same folder
Solution is to provide extra env value with running command LD_LIBRARY_PATH=/data/bin/

/Users/$USER/Library/Android/sdk/platform-tools/adb -s emulator-5566 shell LD_LIBRARY_PATH=/data/bin/ /data/bin/swift-on-android-test

After this all libraties found but there are some missing symbols.

Missing symbol
While running app I got this error.

CANNOT LINK EXECUTABLE "/data/bin/swift-on-android-test": cannot locate symbol "UCNV_FROM_U_CALLBACK_STOP_65" referenced by "/data/bin/libFoundation.so"...

This happen because of --with-library-suffix=swift in icu configure. If we set suffix to lib, it will be set as part of functions name via U_LIB_SUFFIX_C_NAME + U_HAVE_LIB_SUFFIX. Check uvernum.h for more details. Prev it was solved by patching configure, I will try to avoid it by removing suffix

@NikolayJuly
Copy link
Owner Author

NikolayJuly commented Oct 8, 2023

I wanted to compile just icu*.so, and not icu*.so.65.1, but this appears to be more complicated than I anticipated.
Simple renaming of .so files not working in dependencies, becuase SONAME is part of file and linker use it for dependency.
There are 2 possible workarounds:

  • Usepatchelf from this answer. And we can change dependencies and soname
  • We can LD_PRELOAD to preload some .so, which will help during runtime, because needed file already loaded, no need to search for it.

For now I copy all symlinks and actual file with binary and it works for test purpose.

@NikolayJuly
Copy link
Owner Author

This version require installing libxml2 from brew. Looks at StdLibBuild.swift:L65. Ideally we need build it for host because we already checked it out.

@NikolayJuly
Copy link
Owner Author

I will leave this issue not completed. I will create swift-5.7 tag, so later can be found if really needed.
This tag should be buildable on macOS 13 and 14, if you use brew to install libxml2 and autoconf ($ brew install libxml2 and $ brew install autoconf automake libtool)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant