Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update Ubuntu (and therefore Qemu) Version #557

Closed
Alexhuszagh opened this issue May 15, 2021 · 8 comments · Fixed by #591
Closed

Update Ubuntu (and therefore Qemu) Version #557

Alexhuszagh opened this issue May 15, 2021 · 8 comments · Fixed by #591

Comments

@Alexhuszagh
Copy link
Contributor

Alexhuszagh commented May 15, 2021

Currently, on images provided by cross v0.2.1, we get the following after loading into the image:

Cross Information:

$ cross --version
cross 0.2.1
cargo 1.51.0 (43b129a20 2021-03-16)

Image Information

$ $ docker images
REPOSITORY           TAG                                     IMAGE ID       CREATED         SIZE
rustembedded/cross   powerpc-unknown-linux-gnu-0.2.1         e18a73ede16f   10 months ago   752MB
$ docker run -it e18a73ede16f /bin/bash
# lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 16.04.6 LTS
Release:        16.04
Codename:       xenial
# qemu-img --version
qemu-img version 3.0.1
Copyright (c) 2003-2017 Fabrice Bellard and the QEMU Project developers

The latest LTS version for Ubuntu is 20.04, which has newer versions of important dependencies, such as Qemu.

$ docker run -it ubuntu:20.04 /bin/bash
# lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 20.04.2 LTS
Release:        20.04
Codename:       focal
# qemu-img --version
qemu-img version 4.2.1 (Debian 1:4.2-3ubuntu6.16)
Copyright (c) 2003-2019 Fabrice Bellard and the QEMU Project developers

Issue

This actually matters, as numerous bugs, such as for float-point emulation, have been fixed in newer Qemu versions:

In addition, here's a sample code snippet that fails on Qemu 3.0.1 but works on Qemu 4.2.1:

#include <inttypes.h>
#include <stdio.h>
#include <stdint.h>
#include <string.h>

double dn1() {
    double num = 1303.0;
    double den = 1e-20;
    return num * den;
}

double dn2() {
    double num = 1303.0;
    double den = 1e20;
    return num / den;
}


float fn1() {
    float num = 1303.0;
    float den = 1e-20;
    return num * den;
}

float fn2() {
    float num = 1303.0;
    float den = 1e20;
    return num / den;
}

int main() {
  float f = fn1();
  uint32_t bf;
  memcpy(&bf, &f, 8);
  printf("%u\n", bf);

  f = fn2();
  memcpy(&bf, &f, 8);
  printf("%u\n", bf);

  double d = dn1();
  uint64_t bd;
  memcpy(&bd, &d, 8);
  printf("%" PRIu64 "\n", bd);

  d = dn2();
  memcpy(&bd, &d, 8);
  printf("%" PRIu64 "\n", bd);

  return 0;
}

If we run the above using Qemu 3.0.1, we get:

# qemu-ppc --version
qemu-ppc version 3.0.1
Copyright (c) 2003-2017 Fabrice Bellard and the QEMU Project developers
# qemu-ppc ./main-ppc
594566255
594566255
4354430593920867240
4354430593920867241

If we run the above using Qemu 4.2.1, we get:

# qemu-ppc --version
qemu-ppc version 4.2.1 (qemu-4.2.1-1.fc32)
Copyright (c) 2003-2019 Fabrice Bellard and the QEMU Project developers
# qemu-ppc ./main-ppc
594566255
594566255
4354430593920867240
4354430593920867240

In short, we have a rounding error of 1 ULP due to cross using outdated Ubuntu (and therefore Qemu) versions. Ideally, we'd use version 6.0.1 or v5.0.0 (the latter requiring the following PPA, the former requiring compiling from source).

@senden9
Copy link

senden9 commented Jul 20, 2021

Relates to #517

@Alexhuszagh
Copy link
Contributor Author

The system packages for Focal (20.04) are still at 4.2.X, and v6.1.0 of Qemu has recently been released, and there's been numerous bug fixes in 5.X and 6.X. Since we already build from source, it should be trivial to update the local versions and enure all targets build.

@VictorKoenders
Copy link

I'm having an issue while running powerpc64le-unknown-linux-gnu where all floats are 0.0 and are not equal to each other

unknown

Is this related or should I open a separate issue?

@Alexhuszagh
Copy link
Contributor Author

Alexhuszagh commented Apr 18, 2022

@VictorKoenders How recent is the version you are running? It should be fixed in Qemu versions, although I'm not sure a cross-rs release has been published since then, as shown in #586.

@VictorKoenders
Copy link

I think my issue is closer related to #644

@Emilgardis
Copy link
Member

we're using 5.1.0 on upstream cross right now, try using that instead while we get ready for a release.

cargo install cross --git https://github.com/cross-rs/cross

or point the image you're using to the newest (unreleased) image

[target.{{target}}]
image = "ghcr.io/cross-rs/{{target}}:edge"

@Alexhuszagh
Copy link
Contributor Author

Alexhuszagh commented Apr 19, 2022

@VictorKoenders They're related (I know from the Qemu bug tracker): the floating point implementation was somewhat bugged in older versions of Qemu, and pinning it to v3.0.1 (which I did) fixed some issues but causes other ones that existed in 4.x. In Qemu versions 5.x+, (I believe) all of these are fixed. So unpinning the older version of Qemu and using a suitable, modern version should fix these issues (which is done in upstream).

You can test upstream or compile and use a system installation of a modern Qemu version to test. (Right now I'm getting relocation errors on my home PC cross-compiling and don't have time to test).

@Alexhuszagh
Copy link
Contributor Author

@Emilgardis Also this PR is ready but is waiting on the merge of #593, since 2 of the images fail with interactive installed due to tzdata. Any review of that and this PR should be fit to be merged.

Alexhuszagh added a commit to Alexhuszagh/cross that referenced this issue Jun 16, 2022
Alexhuszagh added a commit to Alexhuszagh/cross that referenced this issue Jun 16, 2022
The latest Qemu release contains numerous bug fixes, and other improvements, and the existing patches various issues reported in older Qemu versions. The only changes required are incrementing the Qemu version and adding ninja-build as a temporary dependency, since Qemu now uses it for the build system. However, Ubuntu 16.04, the base image for numerous dependencies we use, still ships with Python3.5, and Qemu 6.1.0 requires Python3.6+, as well as the ninja-build system. We therefore add a check to see if we have a compatible Python version, and if we do, build the last Qemu version, so our code can be fully backwards-compatible.

Fixes cross-rs#557.
Closes cross-rs#587.
Alexhuszagh added a commit to Alexhuszagh/cross that referenced this issue Jun 16, 2022
The latest Qemu release contains numerous bug fixes, and other improvements, and the existing patches various issues reported in older Qemu versions. The only changes required are incrementing the Qemu version and adding ninja-build as a temporary dependency, since Qemu now uses it for the build system. However, Ubuntu 16.04, the base image for numerous dependencies we use, still ships with Python3.5, and Qemu 6.1.0 requires Python3.6+, as well as the ninja-build system. We therefore add a check to see if we have a compatible Python version, and if we do, build the last Qemu version, so our code can be fully backwards-compatible.

Fixes cross-rs#557.
Closes cross-rs#587.
@bors bors bot closed this as completed in 07406ae Jun 16, 2022
bors bot added a commit that referenced this issue Jul 10, 2022
591: Update all GNU docker images to latest LTS version on Docker. r=Emilgardis a=Alexhuszagh

Increment Ubuntu base image versions to 20.04.

Update linux-image script to latest kernel and debian versions.

Update by default to kernel version 5.10.0-8. This means updating our debian source to bullseye from buster. 32-bit big-endian mips was discontinued in bullseye, so we revert to buster. For some images, due to constantly updating linux kernel versions, we need to use wildcards otherwise the build step breaks. Since there may be more than one relevant package, we've added a function to manually expand wildcards and select the best kernel version, `max_kernel_version`. Likewise, on 32-bit big-endian mips, we need to specify the ncurses version.

Created temporary symlinks for autconf and autom4te due to the build expecting a hard-coded version (2.69) of these binaries. Fixed the patch for `debian/rules` due to changed line numbers. Updated the patch to use dwarf rather than sjlj exceptions to patch the template file (`debian/gcc-mingw-w64-i686.install.in`) since `debian/gcc-mingw-w64-i686.install` is overwritten during the build.

For `x86_64-unknown-linux-gnu`, building the linux image fails unless we download specific versions of `libgcc-s1` and `libstdc++6`, since the pre-installed Ubuntu versions are higher than the Debian versions. We therefore extract the specific versions. However, while building the linux image, it prefers these system versions, so we must uninstall them or else while running `qemu-system` it cannot find `libgcc_s1.so.1`. Since `apt` and basically every other package besides `dpkg` relies on `libgcc-s1`, we have to temporarily delete it and reinstall it.

Closes #616.
Closes #557. We've already addressed the Qemu fixes, which will be applied automatically.
Closes #517.
Closes #417.

Replaces #481.

Co-authored-by: Alex Huszagh <ahuszagh@gmail.com>
bors bot added a commit that referenced this issue Jul 16, 2022
591: Update all GNU docker images to latest LTS version on Docker. r=Emilgardis a=Alexhuszagh

Increment Ubuntu base image versions to 20.04.

Update linux-image script to latest kernel and debian versions.

Update by default to kernel version 5.10.0-8. This means updating our debian source to bullseye from buster. 32-bit big-endian mips was discontinued in bullseye, so we revert to buster. For some images, due to constantly updating linux kernel versions, we need to use wildcards otherwise the build step breaks. Since there may be more than one relevant package, we've added a function to manually expand wildcards and select the best kernel version, `max_kernel_version`. Likewise, on 32-bit big-endian mips, we need to specify the ncurses version.

Created temporary symlinks for autconf and autom4te due to the build expecting a hard-coded version (2.69) of these binaries. Fixed the patch for `debian/rules` due to changed line numbers. Updated the patch to use dwarf rather than sjlj exceptions to patch the template file (`debian/gcc-mingw-w64-i686.install.in`) since `debian/gcc-mingw-w64-i686.install` is overwritten during the build.

For `x86_64-unknown-linux-gnu`, building the linux image fails unless we download specific versions of `libgcc-s1` and `libstdc++6`, since the pre-installed Ubuntu versions are higher than the Debian versions. We therefore extract the specific versions. However, while building the linux image, it prefers these system versions, so we must uninstall them or else while running `qemu-system` it cannot find `libgcc_s1.so.1`. Since `apt` and basically every other package besides `dpkg` relies on `libgcc-s1`, we have to temporarily delete it and reinstall it.

Closes #616.
Closes #557. We've already addressed the Qemu fixes, which will be applied automatically.
Closes #517.
Closes #417.

Replaces #481.

Co-authored-by: Alex Huszagh <ahuszagh@gmail.com>
bors bot added a commit that referenced this issue Jul 16, 2022
591: Update all GNU docker images to latest LTS version on Docker. r=Emilgardis a=Alexhuszagh

Increment Ubuntu base image versions to 20.04.

Update linux-image script to latest kernel and debian versions.

Update by default to kernel version 5.10.0-8. This means updating our debian source to bullseye from buster. 32-bit big-endian mips was discontinued in bullseye, so we revert to buster. For some images, due to constantly updating linux kernel versions, we need to use wildcards otherwise the build step breaks. Since there may be more than one relevant package, we've added a function to manually expand wildcards and select the best kernel version, `max_kernel_version`. Likewise, on 32-bit big-endian mips, we need to specify the ncurses version.

Created temporary symlinks for autconf and autom4te due to the build expecting a hard-coded version (2.69) of these binaries. Fixed the patch for `debian/rules` due to changed line numbers. Updated the patch to use dwarf rather than sjlj exceptions to patch the template file (`debian/gcc-mingw-w64-i686.install.in`) since `debian/gcc-mingw-w64-i686.install` is overwritten during the build.

For `x86_64-unknown-linux-gnu`, building the linux image fails unless we download specific versions of `libgcc-s1` and `libstdc++6`, since the pre-installed Ubuntu versions are higher than the Debian versions. We therefore extract the specific versions. However, while building the linux image, it prefers these system versions, so we must uninstall them or else while running `qemu-system` it cannot find `libgcc_s1.so.1`. Since `apt` and basically every other package besides `dpkg` relies on `libgcc-s1`, we have to temporarily delete it and reinstall it.

Closes #616.
Closes #557. We've already addressed the Qemu fixes, which will be applied automatically.
Closes #517.
Closes #417.

Replaces #481.

Co-authored-by: Alex Huszagh <ahuszagh@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants