WIP: Backports 1.6-beta #38949
Merged
WIP: Backports 1.6-beta #38949
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Note there was a discussion whether #38804 should be backported or not, but generally seemed like a good decision to go ahead with it to minimize the difficulty of backporting bugfixes in the future. Wanted to note that here in case it is decided against. |
) MBedTLS fails to build with CMake 3.18.2, due to a bug in CMake 3.18.2 causing failure to find python3. For more information, please see Mbed-TLS/mbedtls#3690 and https://gitlab.kitware.com/cmake/cmake/-/issues/21204. This pull request applies the MBedTLS patch that works around this issue from Mbed-TLS/mbedtls#3691, which has been merged into MBedTLS's development branch but not yet into a tagged release. The latest minor release as of that merge was 2.25.0, so it's probably safe to conjecture that this patch will be live as of minor release 2.26.0; it's also possible that this patch will land on patch releases for 2.24.0 and 2.25.0, but this author doesn't know.
…38969) Without this, our full source dist tarballs still try to download these. Co-authored-by: Elliot Saba <staticfloat@gmail.com>
|
tester_linuxaarch64 failure: Is that one known? |
|
tester_macos64 failures appear to be network failures in the Download tests ("could not connect to server while..." and such). |
Looks like: #38996 (comment) |
(cherry picked from commit c303262)
* Export jl_get_libdir and use it from jl_init to find libjulia. (cherry picked from commit 5cd07f8)
I have no idea whether this is important, but I noticed this while investigating triggers for inference in a package. Feel free to close if you think this is irrelevant. Incidentally, `_artifact_str` is a ~60ms precompile (that's just the inference, not the codegen). I see that its precompile was commented out in 924484f; just mentioning it in case there's an opportunity to squeeze out a bit more latency-reduction. (cherry picked from commit ccaaf56)
Co-authored-by: Julian Samaroo <jpsamaroo@jpsamaroo.me> Co-authored-by: Julian Samaroo <jpsamaroo@jpsamaroo.me> (cherry picked from commit ec386bd)
* [Make] Normalize ppc64le to powerpc64le * [CLI] Fix trampoline on PowerPC 1. Add global entry 2. Don't store `r2` on parent frame 3. Conservative load from toc * remove jl_compile_extern_c from exported funcs (cherry picked from commit 29d5d60)
This doesn't happen that often, (only when `VERSION` changes) but we need to be sensitive to the path to `libjuliainternal.so` changing. (cherry picked from commit 6c047cc)
Hopefully this should preserve the ability to generate these in parallel (and keep a clean tree) and prevent conflicts when updating different deps targets, while consolidating any related files (cherry picked from commit 8af7a04)
libuv's mkdir does not implement the mode argument on Windows, as a result the folder mode is never passed on the Windows platform. Forcibly change the folder permissions to agree with the mode passed into `mkdir`. (cherry picked from commit 556cee7)
(cherry picked from commit ea39b70)
It turns out that aarch64 assembly syntax uses `;` as the comment character, not as the statement separator. So we need to polyfill that. We also set an alignment of 4 bytes (`2^2`) which is required on Apple targets, but is also a good idea on most other aarch64 machines. (cherry picked from commit cdb0833)
Although we were dispatching to the right symbol, the name of the re-exported symbol was lacking its leading underscore. (cherry picked from commit 6937468)
Without this, it was defined in both `libjulia` and `libjulia-internal`, causing `libjulia-internal`'s updates to not be visible. (cherry picked from commit f1a39b6)
JL_DLLEXPORT should be used because it will automatically switch between importing and exporting the symbol as needed. (cherry picked from commit f6b0edc)
Before it was always creating the variables as a pointer even if they are being referenced as a differnt type in the internal library. (cherry picked from commit 8ace36f)
The declaration has been moved to julia.h, which is already included in this file. (cherry picked from commit cb2d476)
(cherry picked from commit 926a3ca)
On Windows, we have a special list of libraries that we search for default symbol resolution. Now that we have symbols defined in `libjulia` and then imported by `libjulia-internal`, we need to make certain that we search `libjulia-internal` first (so that we find non-trampoline functions first) and then `libjulia` (so that we do in fact eventually find the symbols at all). (cherry picked from commit 9489e64)
|
@nanosoldier |
|
Your package evaluation job has completed - possible new issues were detected. A full report can be found here. cc @maleadt |
|
The Activating new environment at `/tmp/jl_qS2Pew/Project.toml`
Updating registry at `~/.julia/registries/General`
Resolving package versions...
Updating `/tmp/jl_qS2Pew/Project.toml`
[e86bdf43] + rr_jll v5.4.1+5
Updating `/tmp/jl_qS2Pew/Manifest.toml`
[692b3bcd] + JLLWrappers v1.1.3
[3576fdfd] + capnproto_jll v0.7.0+1
[e86bdf43] + rr_jll v5.4.1+5
[0dad84c5] + ArgTools
[56f22d72] + Artifacts
[2a0f44e3] + Base64
[ade2ca70] + Dates
[f43a241f] + Downloads
[b77e0a4c] + InteractiveUtils
[b27032c2] + LibCURL
[76f85450] + LibGit2
[8f399da3] + Libdl
[56ddb016] + Logging
[d6f4376e] + Markdown
[ca575930] + NetworkOptions
[44cfe95a] + Pkg
[de0858da] + Printf
[3fa0cd96] + REPL
[9a3f8284] + Random
[ea8e919c] + SHA
[9e88b42a] + Serialization
[6462fe0b] + Sockets
[fa267f1f] + TOML
[a4e569a6] + Tar
[cf7118a7] + UUIDs
[4ec0a83e] + Unicode
[e66e0078] + CompilerSupportLibraries_jll
[deac9b47] + LibCURL_jll
[29816b5a] + LibSSH2_jll
[c8ffd9c3] + MbedTLS_jll
[14a3606d] + MozillaCACerts_jll
[83775a58] + Zlib_jll
[8e850ede] + nghttp2_jll
Precompiling project...
��� Zlib_jll
��� MbedTLS_jll
��� CompilerSupportLibraries_jll
��� JLLWrappers
��� LibSSH2_jll
��� capnproto_jll
��� rr_jll
7 dependencies successfully precompiled in 4 seconds
Resolving package versions...
Updating `/tmp/jl_qS2Pew/Project.toml`
[3161d3a3] + Zstd_jll v1.4.5+2
Updating `/tmp/jl_qS2Pew/Manifest.toml`
[3161d3a3] + Zstd_jll v1.4.5+2
Precompiling project...
��� Zstd_jll
1 dependency successfully precompiled in 1 seconds (7 already precompiled)
ERROR: LoadError: UndefVarError: rr not defined
Stacktrace:
[1] getproperty
@ ./Base.jl:26 [inlined]
[2] rr
@ /buildworker/worker/tester_linux32/build/rr_capture.jl:24 [inlined]
[3] (::var"#1#7")(dir::String)
@ Main /buildworker/worker/tester_linux32/build/rr_capture.jl:25
[4] mktempdir(fn::var"#1#7", parent::String; prefix::String)
@ Base.Filesystem ./file.jl:729
[5] mktempdir(fn::Function, parent::String)
@ Base.Filesystem ./file.jl:727
[6] top-level scope
@ /buildworker/worker/tester_linux32/build/rr_capture.jl:17
in expression starting at /buildworker/worker/tester_linux32/build/rr_capture.jl:17Here's the line that's failing, complaining that rr(func) = Base.invokelatest(rr_jll.rr, func; adjust_LIBPATH=false) |
|
I think that should be fixed by now. |
|
Oh good! :) Thanks. So we just need to wait for the next build? Or is it possible to retry the build or something? |
|
Could we backport 684d5fc ? |
This bump to LibUV adds an important permission to files which should allow them to be deleted more reliably when created in "system"-like locations such as `C:\`. (cherry picked from commit d46198d)
(cherry picked from commit 383db92)
SONAME and RPATH inconsistency resulted in `julia`/`julia-debug` occasionally being unable to find `libjulia.so`, due to `libjulia-internal` not having the right RPATH set (it was set to `$ORIGIN:$ORIGIN/julia`, when it now needs to be set to `$ORIGIN:$ORIGIN/..`). Use `patchelf` to address this during `make install`. We also want to use the SONAME of e.g. `libjulia` as much as possible, so as to avoid searching the filesystem for libraries that are already opened. Also fix a few bugs in building `julia-debug`. (cherry picked from commit 7aa128e)
(cherry picked from commit 176a6e6)
(cherry picked from commit e2f4073)
By naming the section `text` instead of `.text` these sections were mapped into memory without the executable bit set, causing segfaults when attempting to use any trampolines on Windows. (cherry picked from commit b26a386)
There are multiple ways of fixing this, but let's just not let trampolines get re-set after they've already been set. (cherry picked from commit 8981e0b)
If the FDs we allocated are too high, we leaked fds somewhere. On linux it's fairly easy to dump out information about all active FDs, so do that in the failure case, so the logs can help in debugging. (cherry picked from commit 6ae3ded)
By default, I'm having the test suite set up to put any workers spawned with addprocs into their own rr sessions. This doesn't work with SharedArrays, because those workers share memory with the main process. Add a new JULIA_RR environment variable that specifies which rr to use for the test suite and a flag to the testsuite's addprocs command to determine whether or not to detach the workers from the current rr session. (cherry picked from commit ccfbc90)
RR changes the affinity, so 0,1 may not be available in the affinity mask. It's a bit complicated to fix this test and would require parsing the existing affinity mask, so just disable it for now. (cherry picked from commit 2d0211b)
(cherry picked from commit 7a3c153)
(cherry picked from commit 5d40370)
(cherry picked from commit e922744)
(cherry picked from commit 98ff594)
|
#38915 is now approved |
|
I don't want to hold this up, but any chance I could still sneak #39112 into the beta? I am fairly confident that will give us a working Cassette again, which will probably fix quite a few packages. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Backported PRs.
manifest_deps_get#38954 - minor inferrability tweak formanifest_deps_getnothing:methodexpressions #38496 - Update the AST docs onnothing:methodexpressions.ofiles whenVERSIONchanges #38800 - [loader] Must invalidate.ofiles whenVERSIONchangesStdlibArtifacts.tomlfiles #38962 - Source tarballs should containStdlibArtifacts.tomlfilesjl_n_threads#38938 - Fix symbol trampolines for OSX and exportjl_n_threadsDELETEwin ACL patch #39038 - [libuv] Bump to includeDELETEwin ACL patchNon-merged PRs with backport label:
Workerstruct thread-safe. #38405 - Make Distributed.jlWorkerstruct thread-safe.