Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Several issues with partitioning #284

Closed
matthijskooijman opened this issue May 12, 2019 · 1 comment · Fixed by #285
Closed

Several issues with partitioning #284

matthijskooijman opened this issue May 12, 2019 · 1 comment · Fixed by #285

Comments

@matthijskooijman
Copy link
Contributor

I had a look at the image partitioning code and noticed some issues:

  1. There is some empty space between the boot and root partitions. Since commit 40eae05 (make root filesystem 4M aligned (make root filesystem 4M aligned #154)), the start of the root partition is aligned, but the boot partition size is unchanged, which leaves some unused space (which is a waste and makes it slightly harder to add a new partition in the remaining space on a disk, which is why I noticed this).
  2. There is an off-by-one in the aligning of the root partition. If the boot partition size is already exactly rounded to 4M increments, the root partition is put 4M after the boot partition, rather than adjacent to it. This is pretty much Image size is rounded up even when already multiple of 4 MB #156, but about a missing -1 in this line:

    pi-gen/export-image/prerun.sh

    Line 16 in 8acf95f

    ROUNDED_ROOT_SECTOR=$(((2 * BOOT_SIZE + ROUND_SIZE) / ROUND_SIZE * ROUND_SIZE / 512 + 8192))
  3. Sending data into fdisk seems fragile to me. fdisk is, AFAIK, intended to be run interactively and the current script relies on default values and other interactive behaviour which could very well change in future versions.

I had a stab at fixing this with the current fdisk-based approach, which fixes 1) and 2), but obviously not 3). After doing so, I wondered if it would not be better to switch to parted for partition rather than fdisk.

One downside of parted is that, I think, it requires more specific instructions about the partition layout. It has no way to non-interactively say "put root adjacent to boot" without preculculating the exact starting point (note that specifying "use the rest of the disk" is possible by specifying 100%). I suspect this is why fdisk was chosen originally.

However, with the alignment for the root partition that is done now, pretty much the entire partition layout is calculated by prerun.sh already, so it seems sensible to use parted rather than fdisk (parted is already used to print the partition layout, so the dependency is already present).

If this sounds good, I can probably provide a PR to implement this somewhere in the coming days (after that, I'll probably be moving to other projects and might not have time anymore).
@matthijskooijman matthijskooijman mentioned this issue May 13, 2019
Merged
@matthijskooijman
Copy link
Contributor Author

I found that using parted at image creation time would actually prevent a bug at runtime where the PARTUUID of the partitions would change when modifying the partition table using parted. Since that was problematic for me, I went ahead an implemented the switch to parted at image creation time. See #285 for details and the implementation.

matthijskooijman added a commit to matthijskooijman/pi-gen that referenced this issue Jul 7, 2019
@matthijskooijman
Use parted for partitioning
9e20ae8 
Previously, fdisk was used by sending commands into its stdin, which is
not very robust (since it heavily relies on the interactive prompts
offered by fdisk as well as the default values it offers, which seem
prone to changing in future version).

It seems likely that in the past, fdisk was easier than parted since it
provides default values that make it easier to create adjacent
partitions, without precalculating all positions in the script. However
now that partitions are manually being aligned, all data must be
calculated anyway.

This commit changes the partition generation to use parted rather than
fdisk. For this, it rewrites various calculations and renames variables
to be easier to read as well. All values are now in number of bytes,
rather than mixing bytes and sectors.

This commit also makes makes sure that the boot partition and root
partition are always adjacent (previously the root partition was aligned
without also rounding the boot partition size, leaving some empty space
in between).

As a side effect of using parted, this also causes the "bootcode" part
of the MBR to be filled with some default x86 bootcode. This is totally
irrelevant for booting the Raspberry Pi, but it does prevent triggering
a bug in parted. When using parted to change the partition table (e.g.
when resizing the root partition on first boot by raspi-config's
init_resize.sh), the disk identifier would be changed due to this bug,
which would change the PARTUUID of all partitions. The init_resize.sh
script would work around this by updating the PARTUUID in e.g. fstab,
but that's fragile at best.  This commit prevents the bug from
triggering and keeps the disk identifier the same.

See https://debbugs.gnu.org/35714 for details about this parted bug.

This commit fixes RPi-Distro#284.
XECDesign pushed a commit that referenced this issue Sep 25, 2019
@matthijskooijman @XECDesign
Use parted for partitioning (#285)
99f702f 
Previously, fdisk was used by sending commands into its stdin, which is
not very robust (since it heavily relies on the interactive prompts
offered by fdisk as well as the default values it offers, which seem
prone to changing in future version).

It seems likely that in the past, fdisk was easier than parted since it
provides default values that make it easier to create adjacent
partitions, without precalculating all positions in the script. However
now that partitions are manually being aligned, all data must be
calculated anyway.

This commit changes the partition generation to use parted rather than
fdisk. For this, it rewrites various calculations and renames variables
to be easier to read as well. All values are now in number of bytes,
rather than mixing bytes and sectors.

This commit also makes makes sure that the boot partition and root
partition are always adjacent (previously the root partition was aligned
without also rounding the boot partition size, leaving some empty space
in between).

As a side effect of using parted, this also causes the "bootcode" part
of the MBR to be filled with some default x86 bootcode. This is totally
irrelevant for booting the Raspberry Pi, but it does prevent triggering
a bug in parted. When using parted to change the partition table (e.g.
when resizing the root partition on first boot by raspi-config's
init_resize.sh), the disk identifier would be changed due to this bug,
which would change the PARTUUID of all partitions. The init_resize.sh
script would work around this by updating the PARTUUID in e.g. fstab,
but that's fragile at best.  This commit prevents the bug from
triggering and keeps the disk identifier the same.

See https://debbugs.gnu.org/35714 for details about this parted bug.

This commit fixes #284.
PeterJohnson pushed a commit to PeterJohnson/WPILibPi that referenced this issue Dec 7, 2019
@matthijskooijman @PeterJohnson
Use parted for partitioning (RPi-Distro#285)
a236604 
Previously, fdisk was used by sending commands into its stdin, which is
not very robust (since it heavily relies on the interactive prompts
offered by fdisk as well as the default values it offers, which seem
prone to changing in future version).

It seems likely that in the past, fdisk was easier than parted since it
provides default values that make it easier to create adjacent
partitions, without precalculating all positions in the script. However
now that partitions are manually being aligned, all data must be
calculated anyway.

This commit changes the partition generation to use parted rather than
fdisk. For this, it rewrites various calculations and renames variables
to be easier to read as well. All values are now in number of bytes,
rather than mixing bytes and sectors.

This commit also makes makes sure that the boot partition and root
partition are always adjacent (previously the root partition was aligned
without also rounding the boot partition size, leaving some empty space
in between).

As a side effect of using parted, this also causes the "bootcode" part
of the MBR to be filled with some default x86 bootcode. This is totally
irrelevant for booting the Raspberry Pi, but it does prevent triggering
a bug in parted. When using parted to change the partition table (e.g.
when resizing the root partition on first boot by raspi-config's
init_resize.sh), the disk identifier would be changed due to this bug,
which would change the PARTUUID of all partitions. The init_resize.sh
script would work around this by updating the PARTUUID in e.g. fstab,
but that's fragile at best.  This commit prevents the bug from
triggering and keeps the disk identifier the same.

See https://debbugs.gnu.org/35714 for details about this parted bug.

This commit fixes RPi-Distro#284.
general-wedge pushed a commit to HQapp/hq-os that referenced this issue Jan 5, 2020
@matthijskooijman @XECDesign
Use parted for partitioning (RPi-Distro#285)
8dbb713 
Previously, fdisk was used by sending commands into its stdin, which is
not very robust (since it heavily relies on the interactive prompts
offered by fdisk as well as the default values it offers, which seem
prone to changing in future version).

It seems likely that in the past, fdisk was easier than parted since it
provides default values that make it easier to create adjacent
partitions, without precalculating all positions in the script. However
now that partitions are manually being aligned, all data must be
calculated anyway.

This commit changes the partition generation to use parted rather than
fdisk. For this, it rewrites various calculations and renames variables
to be easier to read as well. All values are now in number of bytes,
rather than mixing bytes and sectors.

This commit also makes makes sure that the boot partition and root
partition are always adjacent (previously the root partition was aligned
without also rounding the boot partition size, leaving some empty space
in between).

As a side effect of using parted, this also causes the "bootcode" part
of the MBR to be filled with some default x86 bootcode. This is totally
irrelevant for booting the Raspberry Pi, but it does prevent triggering
a bug in parted. When using parted to change the partition table (e.g.
when resizing the root partition on first boot by raspi-config's
init_resize.sh), the disk identifier would be changed due to this bug,
which would change the PARTUUID of all partitions. The init_resize.sh
script would work around this by updating the PARTUUID in e.g. fstab,
but that's fragile at best.  This commit prevents the bug from
triggering and keeps the disk identifier the same.

See https://debbugs.gnu.org/35714 for details about this parted bug.

This commit fixes RPi-Distro#284.
alexgg pushed a commit to balena-os/pi-gen that referenced this issue Jul 12, 2021
@matthijskooijman @alexgg
Use parted for partitioning (RPi-Distro#285)
1e6949a 
Previously, fdisk was used by sending commands into its stdin, which is
not very robust (since it heavily relies on the interactive prompts
offered by fdisk as well as the default values it offers, which seem
prone to changing in future version).

It seems likely that in the past, fdisk was easier than parted since it
provides default values that make it easier to create adjacent
partitions, without precalculating all positions in the script. However
now that partitions are manually being aligned, all data must be
calculated anyway.

This commit changes the partition generation to use parted rather than
fdisk. For this, it rewrites various calculations and renames variables
to be easier to read as well. All values are now in number of bytes,
rather than mixing bytes and sectors.

This commit also makes makes sure that the boot partition and root
partition are always adjacent (previously the root partition was aligned
without also rounding the boot partition size, leaving some empty space
in between).

As a side effect of using parted, this also causes the "bootcode" part
of the MBR to be filled with some default x86 bootcode. This is totally
irrelevant for booting the Raspberry Pi, but it does prevent triggering
a bug in parted. When using parted to change the partition table (e.g.
when resizing the root partition on first boot by raspi-config's
init_resize.sh), the disk identifier would be changed due to this bug,
which would change the PARTUUID of all partitions. The init_resize.sh
script would work around this by updating the PARTUUID in e.g. fstab,
but that's fragile at best.  This commit prevents the bug from
triggering and keeps the disk identifier the same.

See https://debbugs.gnu.org/35714 for details about this parted bug.

This commit fixes RPi-Distro#284.
UmeshMohan-Dozee pushed a commit to DozeeRnD/pi-gen that referenced this issue Sep 18, 2024
@matthijskooijman @XECDesign
Use parted for partitioning (RPi-Distro#285)
f4e4935 
Previously, fdisk was used by sending commands into its stdin, which is
not very robust (since it heavily relies on the interactive prompts
offered by fdisk as well as the default values it offers, which seem
prone to changing in future version).

It seems likely that in the past, fdisk was easier than parted since it
provides default values that make it easier to create adjacent
partitions, without precalculating all positions in the script. However
now that partitions are manually being aligned, all data must be
calculated anyway.

This commit changes the partition generation to use parted rather than
fdisk. For this, it rewrites various calculations and renames variables
to be easier to read as well. All values are now in number of bytes,
rather than mixing bytes and sectors.

This commit also makes makes sure that the boot partition and root
partition are always adjacent (previously the root partition was aligned
without also rounding the boot partition size, leaving some empty space
in between).

As a side effect of using parted, this also causes the "bootcode" part
of the MBR to be filled with some default x86 bootcode. This is totally
irrelevant for booting the Raspberry Pi, but it does prevent triggering
a bug in parted. When using parted to change the partition table (e.g.
when resizing the root partition on first boot by raspi-config's
init_resize.sh), the disk identifier would be changed due to this bug,
which would change the PARTUUID of all partitions. The init_resize.sh
script would work around this by updating the PARTUUID in e.g. fstab,
but that's fragile at best.  This commit prevents the bug from
triggering and keeps the disk identifier the same.

See https://debbugs.gnu.org/35714 for details about this parted bug.

This commit fixes RPi-Distro#284.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Use parted for partitioning matthijskooijman/pi-gen
1 participant
@matthijskooijman