Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

WIP: Backports for 1.0.5 #33075

Merged
merged 37 commits into from
Aug 31, 2019
Merged

WIP: Backports for 1.0.5 #33075

merged 37 commits into from
Aug 31, 2019

Conversation

KristofferC
Copy link
Member

@KristofferC KristofferC commented Aug 26, 2019

Backported PRs:

Need manual backport:

Contains multiple commits, manual intervention needed:

Non-merged PRs with backport label:

@KristofferC KristofferC added needs nanosoldier run This PR should have benchmarks run on it DO NOT MERGE Do not merge this PR! needs pkgeval Tests for all registered packages should be run with this change release Release management and versioning. labels Aug 26, 2019
JeffBezanson and others added 26 commits August 26, 2019 13:51
(cherry picked from commit 05785f9)
Copy meta nodes from IRCode to CodeInfo

(cherry picked from commit 5168e35)
points to the last non-opt argument of argv, but
in the case where there are only options optind
does not go beyond argc, except on musl libc,
where it becomes argc + 1.

(cherry picked from commit 9de07ce)
Shows the `Dates.format` table which was the behaviour in the Julia 0.6
documentation.

(cherry picked from commit 826bb8b)
This restricts the element type of the input vector to be `<:Real` and
only converts in case the promoted type is concrete.

Fixes #32092.

(cherry picked from commit 587cb82)
This example reproduces the issue observed in issue 32092 on Julia 1.1
that was fixed by PR 32097.

(cherry picked from commit 6da7aa8)
* readdlm(bytearray) shouldn't modify bytearray

* Update stdlib/DelimitedFiles/src/DelimitedFiles.jl

Co-Authored-By: Jeff Bezanson <jeff.bezanson@gmail.com>
(cherry picked from commit 7038210)
* fixes for getindex tab-completion

* test fix

(cherry picked from commit 1eca37e)
This fix adapts the fast path from UInt32(::Char) to the case of digit
parsing.

(cherry picked from commit 510db13)
Statistics uses Random in its tests but doesn't declare it as a test
dependency in its Project.toml.

(cherry picked from commit f6049d6)
This avoids a regression (correctness and performance) caused by #21262
where we were no longer able to fold away the isnull assertion in the
case where the source also contains an explicit isassigned check.

Also upgrade many other CreateInBoundsGEP calls to include the element type (NFC).

(cherry picked from commit a7427aa)
The last return statement is currently referring to a non-existent s
variable. Also fix the related doctest, even though it does not get
tested at the moment.

(cherry picked from commit 7bda2c1)
* bug fixed in read_to_buffer

* added test for #32397.

* switch back to original case

* added torture test for long encoded string randomly laced with spaced

(cherry picked from commit 4b6ab68)
The bug here is a bit subtle, but perhaps best illustrated with
the included test case:
```
function f32579(x::Int64, b::Bool)
    if b
        x = nothing
    end
    if isa(x, Int64)
        y = x
    else
        y = x
    end
    if isa(y, Nothing)
        z = y
    else
        z = y
    end
    return z === nothing
end
```
The code just after SSA conversion looks like:
```
2  1 ─       goto #3 if not _3
3  2 ─ %2  = Main.nothing::Core.Compiler.Const(nothing, false)
5  3 ┄ %3  = φ (#2 => %2, #1 => _2)::Union{Nothing, Int64}
   │   %4  = (%3 isa Main.Int64)::Bool
   └──       goto #5 if not %4
6  4 ─ %6  = π (%3, Int64)
   └──       goto #6
8  5 ─ %8  = π (%3, Nothing)
10 6 ┄ %9  = φ (#4 => %6, #5 => %8)::Union{Nothing, Int64}
   │   %10 = (%9 isa Main.Nothing)::Bool
   └──       goto #8 if not %10
11 7 ─ %12 = π (%9, Nothing)
   └──       goto #9
13 8 ─ %14 = π (%9, Int64)
15 9 ┄ %15 = φ (#7 => %12, #8 => %14)::Union{Nothing, Int64}
   │   %16 = (%15 === Main.nothing)::Bool
   └──       return %16
```
Now, we have special code in SROA (despite it not really being an
SROA transform) that looks at `===` and replaces
it by a nest of phis of booleans. The reasoning for this transform
is that it eliminates a use of a value where we only care about the
type and not the content, thus making it more likely that the value
will subsequently be eligible for SROA. In addition, while it goes
along resolving which values feed into any particular phi, it
accumulates and type conditions it encounters along the way.

Thus in the example above, something like the following happens:
- We look at %14, which πs to %9 with an Int64 constraint, so we only
  consider the #4 predecessor for %9 (due to the constraint), until
  we get to %3, where we again only consider the #1 predecessor,
  where we find the argument (of type Int64) and conclude the result
  is always false
- Now we pop the next item of the stack from our original phi, look
  at %12, which πs to %9 with a Nothing constraint.

At this point we used to terminate the search because we already looked
at %9. However, crucially, we looked at %9 only with an Int64 constraint,
so we missed the fact that `nothing` was in fact a possible value for this
phi. The result was a missing entry in the generated phi node:
```
1 ─       goto #3 if not b
2 ─ %2  = Main.nothing::Core.Compiler.Const(nothing, false)
3 ┄ %3  = φ (#1 => false)::Bool
│   %4  = φ (#2 => %2, #1 => _2)::Union{Nothing, Int64}
│   %5  = (%4 isa Main.Int64)::Bool
└──       goto #5 if not %5
4 ─ %7  = π (%4, Int64)
└──       goto #6
5 ─ %9  = π (%4, Nothing)
6 ┄ %10 = φ (#4 => %3, #5 => %3)::Bool
│   %11 = φ (#4 => %7, #5 => %9)::Union{Nothing, Int64}
│   %12 = (%11 isa Main.Nothing)::Bool
└──       goto #8 if not %12
7 ─       goto #9
8 ─       nothing::Nothing
9 ┄ %16 = φ (#7 => %10, #8 => %10)::Bool
└──       return %16
```
(note the missing #2 predecessor in phi node %3), which would result
in an undefined value at runtime, though in this case LLVM would
have taken advantage of that to just return 0:
```
define i8 @julia_f32579_16051(i64, i8) {
top:
;  @ REPL[1]:15 within `f32579'
  ret i8 0
}
```
Compare this now to the optimized IR with this patch:
```
1 ─       goto #3 if not b
2 ─ %2  = Main.nothing::Core.Compiler.Const(nothing, false)
3 ┄ %3  = φ (#2 => true, #1 => false)::Bool
│   %4  = φ (#2 => %2, #1 => _2)::Union{Nothing, Int64}
│   %5  = (%4 isa Main.Int64)::Bool
└──       goto #5 if not %5
4 ─ %7  = π (%4, Int64)
└──       goto #6
5 ─ %9  = π (%4, Nothing)
6 ┄ %10 = φ (#4 => %3, #5 => %3)::Bool
│   %11 = φ (#4 => %7, #5 => %9)::Union{Nothing, Int64}
│   %12 = (%11 isa Main.Nothing)::Bool
└──       goto #8 if not %12
7 ─       goto #9
8 ─       nothing::Nothing
9 ┄ %16 = φ (#7 => %10, #8 => %10)::Bool
└──       return %16
```
The %3 phi node has its missing entry and the generated LLVM code
correctly returns `b`:
```
define i8 @julia_f32579_16112(i64, i8) {
top:
  %2 = and i8 %1, 1
;  @ REPL[1]:15 within `f32579'
  ret i8 %2
}
```

(cherry picked from commit 0a12944)
* [Statistics] fix type determination in corm

* remove obsolete typeof

* use first element(s) for type initialization

* add test for inhomogeneous data and for overflow

* fix test with NaN

(cherry picked from commit a8a567e)
* fix generic ldiv! for CholeskyPivoted

* add back one empty line to make backport easier

* don't interfere with previous tests

(cherry picked from commit d0e11cf)
dkarrasch and others added 3 commits August 26, 2019 13:51
(cherry picked from commit d8798be)
(cherry picked from commit d128ca1)
helps #15276

(cherry picked from commit db76c21)
(cherry picked from commit 7419f09)
@KristofferC KristofferC force-pushed the backports-release-1.0 branch from dabdea5 to 36ba4f3 Compare August 26, 2019 11:52
Using `futimes` and passing NULL for the `times` parameter sets the current and access file times, but does not require file ownwership.

From man page:
> If times is NULL, the access and modification times are set to the current time. The caller must be the owner of the file, have permission to write the file, or be the superuser.

(cherry picked from commit 3b3a163)
@KristofferC
Copy link
Member Author

@nanosoldier runbenchmarks(ALL, vs = ":release-1.0")

@KristofferC
Copy link
Member Author

This backport PR already has too many commits so I suggest we check if the current set of commits are good and then tag a 1.0.5 based on that. The rest can go in 1.0.6.

@nanosoldier
Copy link
Collaborator

Your benchmark job has completed - possible performance regressions were detected. A full report can be found here. cc @ararslan

@KristofferC
Copy link
Member Author

KristofferC commented Aug 27, 2019

PkgEval: https://gist.github.com/KristofferC/013907eb3c201547009fd34d104fb187

  • ArrayAllez
  • BenchmarkProfiles
  • ContinuousTransformations
  • DiffEqBiological
  • EponymTuples
  • IterableTables
  • Reactive
  • StaticNumbers

Everything passes locally except for Reactive seems to be a tol error Evaluated: isapprox(1.567024099315486e9, 1.567024099326614e9; atol=0.01)

musm and others added 6 commits August 27, 2019 17:22
* Add Float16 comparisons

* Add @eval

* Add union

* Add != to tests

(cherry picked from commit 1d3c371)
* proper diagonal in copytri! (fix #30055)

* added sprandn methods with Type

* additional parameter in copytri! for diagonal

* @inline copytri! to enforce constant propagation

(cherry picked from commit 4be9339)
* Fix incorrect sign in atanh

In the case that x==-1, we have to flip the sign of ξ.

* Formatting: Add space after comma

Co-Authored-By: cafaxo <cafaxo@gmail.com>

* Add test

* Do not drop sign of zero

* Flip sign to avoid a DomainError

This fixes atanh(prevfloat(-1.0) + 0im)

* Test all four combinations

Co-Authored-By: cafaxo <cafaxo@gmail.com>

* Tests for expressions that were signed incorrectly

* Use the correct type for 1

* Update doc

* Update doc: Remove trailing whitespace

(cherry picked from commit 66341a3)
@KristofferC KristofferC force-pushed the backports-release-1.0 branch from ec7c6f6 to cad5fc6 Compare August 27, 2019 15:25
@KristofferC KristofferC removed the needs nanosoldier run This PR should have benchmarks run on it label Aug 27, 2019
@KristofferC KristofferC force-pushed the backports-release-1.0 branch from cad5fc6 to fc33508 Compare August 28, 2019 07:18
@KristofferC KristofferC force-pushed the backports-release-1.0 branch from fc33508 to b717256 Compare August 28, 2019 20:09
@ararslan ararslan removed DO NOT MERGE Do not merge this PR! needs pkgeval Tests for all registered packages should be run with this change labels Aug 31, 2019
@ararslan ararslan merged commit 712150c into release-1.0 Aug 31, 2019
@delete-merged-branch delete-merged-branch bot deleted the backports-release-1.0 branch August 31, 2019 18:18
vchuravy pushed a commit to JuliaLang/Distributed.jl that referenced this pull request Oct 6, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
release Release management and versioning.
Projects
None yet
Development

Successfully merging this pull request may close these issues.