Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add elementary 7 daily images #565

Merged
merged 6 commits into from
Nov 4, 2021
Merged

Add elementary 7 daily images #565

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
e58cfd9
Add elementary 7 daily images
davidmhewitt Oct 25, 2021
17ea6e7
Fix debian container version
davidmhewitt Oct 25, 2021
4eec094
Switch back to md5 checksums
davidmhewitt Oct 25, 2021
b8d8856
Switch 6.0 workflow to debian:latest
davidmhewitt Oct 25, 2021
781a3ce
Improve key getting process
davidmhewitt Oct 26, 2021
380af7c
Update debian container names
davidmhewitt Oct 26, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/daily-6.0.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ jobs:
runs-on: ubuntu-latest

container:
image: debian:buster
image: debian:latest
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updating to the latest stable release of Debian lets us drop a couple of patches that I'd upstreamed against live-build. Because these patches are dropped from the build.sh script, which is used for building all versions of the ISO, we need to use this newer container everywhere.

volumes:
- /proc:/proc
options: --privileged
Expand Down
27 changes: 27 additions & 0 deletions .github/workflows/daily-7.0.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
name: daily-7.0

on:
push:
branches:
- master
schedule:
- cron: "0 0 * * *"
workflow_dispatch: {}

jobs:
build:
runs-on: ubuntu-latest

container:
image: debian:latest
volumes:
- /proc:/proc
options: --privileged

steps:
- name: Clone build scripts
uses: actions/checkout@v1

- name: Build and upload daily .iso
run: |
./workflows.sh etc/terraform-daily-7.0-azure.conf "${{ secrets.key }}" "${{ secrets.secret }}" "${{ secrets.endpoint }}" "${{ secrets.bucket }}"
2 changes: 1 addition & 1 deletion .github/workflows/stable.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ jobs:
runs-on: ubuntu-latest

container:
image: debian:buster
image: debian:latest
volumes:
- /proc:/proc
options: --privileged
Expand Down
2 changes: 1 addition & 1 deletion README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@ Configure the channel in the `etc/terraform.conf` (stable, daily), then run:
docker run --privileged -i -v /proc:/proc \
-v ${PWD}:/working_dir \
-w /working_dir \
debian:buster \
debian:latest \
/bin/bash -s etc/terraform.conf < build.sh
```

Expand Down
18 changes: 12 additions & 6 deletions build.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -24,20 +24,26 @@ echo -e "
"

apt-get update
apt-get install -y live-build patch ubuntu-keyring
apt-get install -y live-build patch gnupg2 binutils zstd

# TODO: Remove once live-build is able to acommodate for cases where LB_INITRAMFS is not live-boot:
# https://salsa.debian.org/live-team/live-build/merge_requests/31
patch -d /usr/lib/live/build/ < live-build-fix-syslinux.patch
# The Debian repositories don't seem to have the `ubuntu-keyring` or `ubuntu-archive-keyring` packages
# anymore, so we add the archive keys manually. This may need to be updated if Ubuntu changes their signing keys
# To get the current key ID, find `ubuntu-keyring-xxxx-archive.gpg` in /etc/apt/trusted.gpg.d on a running
# system and run `gpg --keyring /etc/apt/trusted.gpg.d/ubuntu-keyring-xxxx-archive.gpg --list-public-keys `
apt-key adv --recv-keys --keyserver keyserver.ubuntu.com F6ECB3762474EDA9D21B7022871920D1991BC93C

# TODO: This patch was submitted upstream at:
# https://salsa.debian.org/live-team/live-build/-/merge_requests/255
# This can be removed when our Debian container has a version containing this fix
patch -d /usr/lib/live/build/ < live-build-fix-shim-remove.patch

# TODO: Remove this once debootstrap 1.0.117 or newer is released and available:
# TODO: This can be removed when our Debian container has debootstrap 1.0.124 or later
# It's needed to support the new zstd .deb package compression that Ubuntu is doing
patch -d /usr/share/debootstrap/ < debootstrap-backport-zstd-support.patch
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ubuntu have switched to using zstd compression for .deb packages in impish+ , the support for bootstrapping from zstd compressed packages isn't in the stable version of Debian, so we backport it here.


# TODO: Remove this once debootstrap has a script to build jammy images in our container:
# https://salsa.debian.org/installer-team/debootstrap/blob/master/debian/changelog
ln -sfn /usr/share/debootstrap/scripts/gutsy /usr/share/debootstrap/scripts/focal
ln -sfn /usr/share/debootstrap/scripts/gutsy /usr/share/debootstrap/scripts/jammy
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The focal symlink now exists upstream and is just a symlink to gutsy.

jammy doesn't exist upstream yet, so we symlink that to gutsy too.


build () {
BUILD_ARCH="$1"
Expand Down
19 changes: 19 additions & 0 deletions debootstrap-backport-zstd-support.patch
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
--- /usr/share/debootstrap/functions.orig 2021-10-23 23:13:10.576805331 +0000
+++ /usr/share/debootstrap/functions 2021-10-23 23:14:05.465350379 +0000
@@ -974,6 +974,7 @@
case "$tarball" in
control.tar.gz) cat_cmd=zcat ;;
control.tar.xz) cat_cmd=xzcat ;;
+ control.tar.zst) cat_cmd=zstdcat ;;
control.tar) cat_cmd=cat ;;
*) error 1 UNKNOWNCONTROLCOMP "Unknown compression type for %s in %s" "$tarball" "$pkg" ;;
esac
@@ -996,6 +997,7 @@
data.tar.gz) cat_cmd=zcat ;;
data.tar.bz2) cat_cmd=bzcat ;;
data.tar.xz) cat_cmd=xzcat ;;
+ data.tar.zst) cat_cmd=zstdcat ;;
data.tar) cat_cmd=cat ;;
*) error 1 UNKNOWNDATACOMP "Unknown compression type for %s in %s" "$tarball" "$pkg" ;;
esac

2 changes: 2 additions & 0 deletions etc/auto/config
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,8 @@ lb config noauto \
--linux-packages linux-image \
--linux-flavours "$KERNEL_FLAVORS" \
--bootappend-live "boot=casper maybe-ubiquity quiet splash" \
--debootstrap-options="--extractor=ar --keyring=/etc/apt/trusted.gpg" \
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use ar to extract packages which allows the backported zstd support.

Have debootstrap trust the ubuntu keyring we imported in build.sh.

--checksums md5 \
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

live-build switch to SHA256 sums for the iso integrity check by default in this version of Debian.

The Ubuntu live iso integrity check on booting the live CD still uses MD5, so switch that over here or else we don't get the integrity check on startup.

--mirror-bootstrap "$MIRROR_URL" \
--parent-mirror-bootstrap "$MIRROR_URL" \
--mirror-chroot-security "http://security.ubuntu.com/ubuntu/" \
Expand Down
9 changes: 0 additions & 9 deletions etc/config/package-lists.default/pool.list.binary
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,12 +4,10 @@ dkms
intel-microcode
iucode-tool
lupin-support
mouseemu
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No longer exists in jammy, seems to be an ancient package for supporting emulated middle/right buttons on PowerPC based Macs.

Packages in this list aren't even installed anyway, they're just put on the pool on the apt-cdrom repository in case something without internet access wants to install them.

setserial
user-setup

efibootmgr
grub-efi
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We have the architecture specific version of grub-efi below. Removing this prevents confusion as to what architecture ends up in the pool on the iso.

secureboot-db

#if ARCHITECTURES amd64
Expand All @@ -19,10 +17,3 @@ grub-efi-amd64-signed
shim
shim-signed
#endif

#if ARCHITECTURES i386
grub-efi-ia32
grub-efi-ia32-bin
sl-modem-daemon
#endif

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We don't build i386 images.

36 changes: 36 additions & 0 deletions etc/terraform-daily-7.0-azure.conf
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
# target architecture - i386, amd64 or all
ARCH="amd64"

# base codename
BASECODENAME="jammy"

# base version
BASEVERSION="22.04"

# distribution codename
CODENAME="next"

# distribution version
VERSION="7.0"

# distribution channel
CHANNEL="daily"

# distribution name
NAME="elementary OS"

# mirror to fetch packages from
MIRROR_URL="http://azure.archive.ubuntu.com/ubuntu/"

# use HWE kernel and packages?
HWE_KERNEL="no"
HWE_X11="no"

# use appcenter ppa
INCLUDE_APPCENTER=""

# suffix for generated .iso files
OUTPUT_SUFFIX=""

# folder suffix for the package lists to use
PACKAGE_LISTS_SUFFIX="default"
11 changes: 5 additions & 6 deletions live-build-fix-shim-remove.patch
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,16 +1,15 @@
--- /usr/lib/live/build/binary_grub-efi 2019-03-11 10:05:40.000000000 +0000
+++ /usr/lib/live/build/binary_grub-efi_v2 2021-08-04 13:37:20.064547041 +0000
@@ -267,8 +267,12 @@
--- binary_grub-efi 2021-04-02 15:43:54.000000000 +0000
+++ binary_grub-efi_v2 2021-10-23 22:43:57.314540341 +0000
@@ -280,8 +280,12 @@
# Saving cache
Save_cache cache/packages.binary
Save_package_cache binary

- # Removing depends
- Remove_package
+ # Removing depends, some bootloader packages are marked as Protected/Important
+ # in Ubuntu, so temporarily add an apt flag to allow them to be removed
+ PRE_APT_OPTIONS="${APT_OPTIONS}"
+ APT_OPTIONS="${APT_OPTIONS} --allow-remove-essential"
+ Remove_package
Remove_packages
+ APT_OPTIONS="${PRE_APT_OPTIONS}"
;;

Expand Down
17 changes: 0 additions & 17 deletions live-build-fix-syslinux.patch
View file
Open in desktop

This file was deleted.