WIP: Backports for 1.3-RC2 #32973

KristofferC · 2019-08-20T08:34:59Z

Backported PRs:

Disable BB GMP and MPFR, to fix performance regressions #32102 - Disable BB GMP and MPFR, to fix performance regressions
Add missing seed!(::_GLOBAL_RNG) method #32961 - Add missing seed!(::_GLOBAL_RNG) method
Bump Pkg version. #32975 - Bump Pkg version.
Fix AbstractQ conversion to different-typed matrix #32979 - Fix AbstractQ conversion to different-typed matrix
docs: prepend v to tag string #32993 - docs: prepend v to tag string
fix some broadcast patterns to avoid allocations #33007 - fix some broadcast patterns to avoid allocations
Clarify the behavior of rand/randn for complex types #33002 - Clarify the behavior of rand/randn for complex types
Bump libgit2 to 0.28.2 #32806 - Bump libgit2 to 0.28.2
Disallow var syntax in string interpolation #32948 - Disallow var syntax in string interpolation
Add missing checksums for libgit2 0.28 #33022 - Add missing checksums for libgit2 0.28
Fix incorrect use of isimmutable #33042 - Fix incorrect use of isimmutable
Use pivoting as the default in LU regardless of the element type. #32989 - Use pivoting as the default in LU regardless of the element type.
improve performance of setindex! on IdDict #33009 - improve performance of setindex! on IdDict
Fix LU doctest broken by #32989. #33051 - Fix LU doctest broken by Use pivoting as the default in LU regardless of the element type. #32989.
improve precompilation coverage #33006 - improve precompilation coverage
fix #32903, regression in Ctrl-C after adding io locks #33031 - fix REPL sometimes breaks after printing is interrupted #32903, regression in Ctrl-C after adding io locks
fix performance regression in broadcasting in some cases #33079 - fix performance regression in broadcasting in some cases
Upgrade doc/make.jl to introspect branch/tag names from git info #32823 - Upgrade doc/make.jl to introspect branch/tag names from git info
Update unwind BB binaries to include binaries for musl libc. #33032 - Update unwind BB binaries to include binaries for musl libc.
fix #32970, at-threads disabled after a loop errors #33034 - fix Threads are disabled if an error occurs #32970, at-threads disabled after a loop errors
error instead of widening index types in sparse #33083 - error instead of widening index types in sparse
Fix Float16 conversion from large Float32s #33084 - Fix Float16 conversion from large Float32s
fix #33020, check axes for broadcasted assignment from tuples #33080 - fix Julia crashes when DataFrames emits a setproperty! deprecation warning #33020, check axes for broadcasted assignment from tuples
Workaround for performance penalty of splatting a number #33108 - Workaround for performance penalty of splatting a number
Fix Float16 conversion from large Float32s #33084 - Fix Float16 conversion from large Float32s
Bump libssh2 to v1.9.0, use MbedTLS on all platforms #33089 - Bump libssh2 to v1.9.0, use MbedTLS on all platforms
ccall: fix vararg count computation #33123 - ccall: fix vararg count computation
Fix which behavior when passed an empty string #33150 - Fix which behavior when passed an empty string
fix #33135, wrong scope resolution with conflicting sparams and let vars #33145 - fix inconsistent error for let bound variables conflicting with static parameters #33135, wrong scope resolution with conflicting sparams and let vars
Replace realpath implementation with libuv #33116 - Replace realpath implementation with libuv
RNG: fix Sampler selection for GLOBAL_RNG #33172 - RNG: fix Sampler selection for GLOBAL_RNG
Fix assert_havelock(::ReentrantLock) to assert that the _current-task_ has the lock. #33159 - Fix assert_havelock(::ReentrantLock) to assert that the current-task has the lock.
Factor-out logic to determine the path of the precompilation cache file. #33173 - Factor-out logic to determine the path of the precompilation cache file.
Bump Pkg for 1.3.0-rc2. #33180 - Bump Pkg for 1.3.0-rc2.
Normalize index to CartesianIndex in _modify! #33187 - Normalize index to CartesianIndex in _modify!

Non-merged PRs with backport label:

Update GMP and MPFR builds to use better compiler flags #33096 - Update GMP and MPFR builds to use better compiler flags

(cherry picked from commit 8ebe564)

KristofferC · 2019-08-20T08:36:09Z

Since we didn't run benchmarks for 1.2-RC1, running it immediately here instead.

@nanosoldier runbenchmarks(ALL, vs = ":release-1.2")

nanosoldier · 2019-08-20T15:35:31Z

Your benchmark job has completed - possible performance regressions were detected. A full report can be found here. cc @ararslan

(cherry picked from commit 64be75f)

(cherry picked from commit fcf9063)

(cherry picked from commit 6723492)

(cherry picked from commit 191f2ae)

(cherry picked from commit fa641c6)

This allows dropping MbedTLS patches which have been upstreamed. The order in which configuration options are returned has changed, making a test fail: make the code more robust by giving priority to more specific options over global ones. (cherry picked from commit d0b5d98)

The var"##" syntax should be disabled in string interpolation. Disallow `var` syntax in command interpolations This is special cased for compatibility. A more general fix would be to make cmd interpolation syntax exactly the same as string interpolation. (cherry picked from commit 050160c)

(cherry picked from commit cd16f6e)

This function operates on values not on types (though it is a bit of a trap). Also add a test to catch this bug. (cherry picked from commit aee3fc2)

…2989) For types that weren't subtypes of AbstractFloat, we used to try to LU factorize without pivoting and only use pivoting when it failed. This caused large numerical errors when computing the LU for element types which promoted to float like numbers such as most integers. The behavior was never documented and is error prone. Hence, this PR removes the behavior. (cherry picked from commit 5af3c2a)

(cherry picked from commit d1979e3)

(cherry picked from commit 24f1611)

(cherry picked from commit 4c12b7a)

- move the place where --trace-compile outputs precompile statement to a location that catches more cases - tweak the REPL code to be more amenable to precompilation in light of - instead of trying to encode all the rules where the precompile emitter fails (#28808) just try to precompile and do nothing if it fails. (cherry picked from commit c0478d8)

(cherry picked from commit 7d65e13)

(cherry picked from commit 56fdb19)

Upgrade `doc/make.jl` to introspect branch/tag names from git info (cherry picked from commit ad8cf8d)

(cherry picked from commit e6dd72f)

(cherry picked from commit 5c42f10)

Followup to https://github.com/JuliaLang/julia/pull/31724/files#r317686891; instead of widening the index type dynamically based upon the index vector length, just throw an error in the case where the index type cannot hold all the indices in a CSC format. This previously was an OOB access (and likely segfault) in 1.2, so throwing an error here is not a breaking change -- and throwing an error restores type stability, addressing the performance regression flagged in #32985. (cherry picked from commit 9725fb4)

KristofferC · 2019-08-27T07:37:26Z

@nanosoldier runbenchmarks(ALL, vs = ":release-1.2")

nanosoldier · 2019-08-27T14:32:54Z

Your benchmark job has completed - possible performance regressions were detected. A full report can be found here. cc @ararslan

KristofferC · 2019-08-27T19:24:36Z

KristofferC · 2019-09-05T14:48:58Z

SymbolServer passes on master (of the package)

KristofferC · 2019-09-05T14:54:15Z

SymEngine: something with the LU pivoting that the package cant handle: symengine/SymEngine.jl#177

KristofferC · 2019-09-06T06:37:40Z

ApproxFunFourier passes locally

KristofferC · 2019-09-06T07:02:19Z

ApproxFunSingularities passes locally.

KristofferC · 2019-09-06T07:02:55Z

BSONqs fixed by richiejp/BSONqs.jl@37e5af5.

KristofferC · 2019-09-06T09:15:49Z

ForwardDiff is some small numeric error. Passes locally with JuliaDiff/ForwardDiff.jl#409.

KristofferC · 2019-09-06T09:32:30Z

I think we should merge this and tag an RC2.

oxinabox · 2019-09-06T09:34:52Z

I think we should merge this and tag an RC2.

Conceptually, shouldn't every release candidate be something we think could be the release final?
I see the value in adding another that is part way there to check things are fixed, that we think are
but calling it a release candidate when there are half a dozen known blockers is not really very clear.

KristofferC · 2019-09-06T09:40:15Z

What are the half dozen known blockers?

oxinabox · 2019-09-06T09:42:14Z

I see about half-a-dozen missing ticks in
#32973 (comment)
or is that just out of date?

KristofferC · 2019-09-06T09:49:17Z

The ambiguities are not release-blockers.
JuMP error is reported at Heads up about test error in JuMP for upcoming 1.3 RC-2. jump-dev/JuMP.jl#2054.
TravelingSalesmanExact already had big inference problems.
Alpine is unclear since it passes locally in some cases.
CBinding apparently uses ccall erroneously according to Jameson
rdiv with transpose of LU on RHS no longer works #33177 is a real issue for Nabla

The point of the release process is to get a good release in the end and it requires a bit of pragmatism.

I can just check the boxes for you if that will make you happy?

(cherry picked from commit 4c8cd3b)

@Spawn

…sk_ has the lock. (#33159) * Fix `assert_havelock(::ReentrantLock)` to assert that the _current-task_ has the lock. Before this commit, new threads would incorrectly believe that they held a lock on a Condition when they actually didn't, and would allow illegal operations, e.g. notify: ```julia julia> c = Threads.Condition() Base.GenericCondition{ReentrantLock}(Base.InvasiveLinkedList{Task}(nothing, nothing), ReentrantLock(nothing, Base.GenericCondition{Base.Threads.SpinLock}(Base.InvasiveLinkedList{Task}(nothing, nothing), Base.Threads.SpinLock(Base.Threads.Atomic{Int64}(0))), 0)) julia> lock(c) julia> fetch(Threads.@Spawn Base.assert_havelock(c)) # This should be an ERROR (the new thread doesn't have the lock) julia> fetch(Threads.@Spawn notify(c)) # This should be an ERROR (the new thread doesn't have the lock) 0 julia> fetch(Threads.@Spawn wait(c)) # This error should be caught earlier (in assert_havelock). ERROR: TaskFailedException: unlock from wrong thread Stacktrace: [1] error(::String) at ./error.jl:33 [2] unlockall(::ReentrantLock) at ./lock.jl:121 [3] wait(::Base.GenericCondition{ReentrantLock}) at ./condition.jl:105 [4] (::var"##19#20")() at ./threadingconstructs.jl:113 ``` (The same holds for `@async` as `@spawn`.) After this change, the assertion works correctly: ``` julia> c = Threads.Condition(); julia> lock(c) julia> fetch(Threads.@Spawn Base.assert_havelock(c)) # This correctly ERRORs ERROR: TaskFailedException: concurrency violation detected Stacktrace: [1] error(::String) at ./error.jl:33 [2] concurrency_violation() at ./condition.jl:8 [3] assert_havelock at ./condition.jl:28 [inlined] [4] assert_havelock at ./REPL[22]:1 [inlined] [5] assert_havelock(::Base.GenericCondition{ReentrantLock}) at ./condition.jl:73 [6] (::var"##21#22")() at ./threadingconstructs.jl:113 ``` Also adds unit test that failed before this commit but now succeeds * Remove default impl of `assert_havelock`; add `::SpinLock` impl (cherry picked from commit 784eb57)

(cherry picked from commit 2de79f7)

oxinabox · 2019-09-06T09:53:29Z

🤷‍♂
Ticking the boxes would make me less happy.

This reverts commit a7fabc9.

This reverts commit 8d8b3d9.

KristofferC · 2019-09-06T10:45:39Z

No matter if we call the merged of this RC2 or not, I will merge this because this PR is getting unwieldy and it needs a fresh start.

ericphanson · 2019-09-06T12:35:49Z

I think the TravelingSalesmanExact timeouts are the same issue as #32167 (ie MOI v0.9 has way slower inference on 1.2 than 1.1). I don’t think anyone uses my package but it’s just a pretty simple JuMP problem and I think it might reflect real regressions that do affect users (who likely haven’t decided to turn simple JuMP solves into a package, so it doesn’t show up on PkgEval). But I guess that’s a 1.2 issue rather than a 1.3 one directly, and I could use compat to restrict to MOI v0.8.

Edit: hmm, the problem persists with MOI v0.8: https://travis-ci.com/ericphanson/TravelingSalesmanExact.jl/builds/126257518

(cherry picked from commit f71f58e)

(cherry picked from commit 6a20ad7)

KristofferC · 2019-09-08T14:54:15Z

Yeah, figuring that out would be very nice. It is usually hard to do MWEs for these type of issues though.

Add missing seed!(::_GLOBAL_RNG) method (#32961)

91ffdca

(cherry picked from commit 8ebe564)

KristofferC added needs nanosoldier run This PR should have benchmarks run on it DO NOT MERGE Do not merge this PR! needs pkgeval Tests for all registered packages should be run with this change labels Aug 20, 2019

KristofferC mentioned this pull request Aug 20, 2019

Benchmark regressions of 1.3 vs 1.2 #32985

Closed

fredrikekre and others added 20 commits August 25, 2019 10:14

Bump Pkg version. (#32975)

1a19ae8

(cherry picked from commit 64be75f)

Fix AbstractQ conversion to different-typed matrix (#32979)

b0d635e

(cherry picked from commit fcf9063)

docs: prepend v to tag string (#32993)

2a1ab32

(cherry picked from commit 6723492)

fix and test that a certain broadcast pattern doesn't allocate (#33007)

f482cb4

(cherry picked from commit 191f2ae)

Clarify the behavior of rand/randn for complex types (#33002)

222fa90

(cherry picked from commit fa641c6)

Add missing checksums for libgit2 0.28 (#33022)

438632e

(cherry picked from commit cd16f6e)

Fix incorrect use of isimmutable (#33042)

add5c78

This function operates on values not on types (though it is a bit of a trap). Also add a test to catch this bug. (cherry picked from commit aee3fc2)

improve performance of setindex! on IdDict (#33009)

0905f7c

(cherry picked from commit d1979e3)

Fix LU doctest broken by #32989. (#33051)

aac65c2

(cherry picked from commit 24f1611)

Disable BB GMP and MPFR, to fix performance regressions

d7187e4

(cherry picked from commit 4c12b7a)

fix #32903, regression in Ctrl-C after adding io locks (#33031)

8e66361

(cherry picked from commit 7d65e13)

fix performance regression in broadcasting in some cases (#33079)

27a12c8

(cherry picked from commit 56fdb19)

Upgrade doc/make.jl to introspect branch/tag names from git… (#32823)

c0c8038

Upgrade `doc/make.jl` to introspect branch/tag names from git info (cherry picked from commit ad8cf8d)

Update unwind BB binaries for musl libc. (#33032)

dbcc3dd

(cherry picked from commit e6dd72f)

fix #32970, at-threads disabled after a loop errors (#33034)

9f39733

(cherry picked from commit 5c42f10)

ericphanson mentioned this pull request Sep 6, 2019

Don’t use internal JuMP fields ericphanson/TravelingSalesmanExact.jl#6

Closed

rfourquet and others added 3 commits September 6, 2019 05:52

RNG: fix Sampler selection for GLOBAL_RNG (#33172)

3e543a2

(cherry picked from commit 4c8cd3b)

Factor-out logic to determine the path of the precompilation cache file.

8f662b5

(cherry picked from commit 2de79f7)

Kristoffer Carlsson added 2 commits September 6, 2019 06:01

Revert "Fix repr on Period Types, and DateTime, Date (#30817)"

a5539a0

This reverts commit a7fabc9.

Revert "Fix repr on DateTime (#30200)"

c63f70f

This reverts commit 8d8b3d9.

KristofferC removed DO NOT MERGE Do not merge this PR! needs nanosoldier run This PR should have benchmarks run on it needs pkgeval Tests for all registered packages should be run with this change labels Sep 6, 2019

fredrikekre and others added 2 commits September 8, 2019 03:24

Bump Pkg for 1.3.0-rc2. (#33180)

965fc11

(cherry picked from commit f71f58e)

Normalize index to CartesianIndex in _modify! (#33187)

e69ce0f

(cherry picked from commit 6a20ad7)

KristofferC merged commit 58803fe into release-1.3 Sep 8, 2019

delete-merged-branch bot deleted the backports-release-1.3 branch September 8, 2019 14:53

KristofferC mentioned this pull request Sep 19, 2019

Backports for Julia 1.3-RC3 #33221

Merged

18 tasks

omus mentioned this pull request Mar 10, 2020

Support git credential helper resetting #35064

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

WIP: Backports for 1.3-RC2 #32973

WIP: Backports for 1.3-RC2 #32973

KristofferC commented Aug 20, 2019 •

edited

Loading

KristofferC commented Aug 20, 2019

nanosoldier commented Aug 20, 2019

KristofferC commented Aug 27, 2019

nanosoldier commented Aug 27, 2019

KristofferC commented Aug 27, 2019 •

edited

Loading

KristofferC commented Sep 5, 2019

KristofferC commented Sep 5, 2019

KristofferC commented Sep 6, 2019

KristofferC commented Sep 6, 2019

KristofferC commented Sep 6, 2019

KristofferC commented Sep 6, 2019

KristofferC commented Sep 6, 2019

oxinabox commented Sep 6, 2019

KristofferC commented Sep 6, 2019

oxinabox commented Sep 6, 2019 •

edited

Loading

KristofferC commented Sep 6, 2019 •

edited

Loading

oxinabox commented Sep 6, 2019

KristofferC commented Sep 6, 2019

ericphanson commented Sep 6, 2019 •

edited

Loading

KristofferC commented Sep 8, 2019

WIP: Backports for 1.3-RC2 #32973

WIP: Backports for 1.3-RC2 #32973

Conversation

KristofferC commented Aug 20, 2019 • edited Loading

KristofferC commented Aug 20, 2019

nanosoldier commented Aug 20, 2019

KristofferC commented Aug 27, 2019

nanosoldier commented Aug 27, 2019

KristofferC commented Aug 27, 2019 • edited Loading

KristofferC commented Sep 5, 2019

KristofferC commented Sep 5, 2019

KristofferC commented Sep 6, 2019

KristofferC commented Sep 6, 2019

KristofferC commented Sep 6, 2019

KristofferC commented Sep 6, 2019

KristofferC commented Sep 6, 2019

oxinabox commented Sep 6, 2019

KristofferC commented Sep 6, 2019

oxinabox commented Sep 6, 2019 • edited Loading

KristofferC commented Sep 6, 2019 • edited Loading

oxinabox commented Sep 6, 2019

KristofferC commented Sep 6, 2019

ericphanson commented Sep 6, 2019 • edited Loading

KristofferC commented Sep 8, 2019

KristofferC commented Aug 20, 2019 •

edited

Loading

KristofferC commented Aug 27, 2019 •

edited

Loading

oxinabox commented Sep 6, 2019 •

edited

Loading

KristofferC commented Sep 6, 2019 •

edited

Loading

ericphanson commented Sep 6, 2019 •

edited

Loading