Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

vz: configure Rosetta’s ahead of time (AOT) caching options using an abstract socket #2489

Conversation

norio-nomura
Copy link
Contributor

This change allows starting the /mnt/lima-rosetta/rosttad daemon in the VM. Without this change, it cannot be started as shown below:

$ sudo /mnt/lima-rosetta/rosettad daemon
rosetta error: Failed bind socket: 98
Trace/breakpoint trap

Since rosettad needs to run in the same namespace as rosetta and must be executed directly from /mnt/lima-rosetta/rosettad (copies of rosettad do not work), it is difficult to use during container image builds or similar processes.

In a container, rosettad can be run as follows:

$ docker run -it --rm --platform linux/amd64 --name test-rosettad -v/mnt/lima-rosetta:/mnt/lima-rosetta ubuntu
root@45f887cb0eaa:/# ls -la ~/.cache/rosettad
/usr/bin/ls: cannot access '/root/.cache/rosettad': No such file or directory
root@45f887cb0eaa:/# /mnt/lima-rosetta/rosettad daemon&
[1] 10
root@45f887cb0eaa:/# ls -la ~/.cache/rosettad
total 2700
drwxr-xr-x 2 root root    4096 Jul 17 04:16 .
drwxr-xr-x 3 root root    4096 Jul 17 04:16 ..
-rwxr-xr-x 1 root root  186600 Jul 17 04:16 169a47096736cdeb5714fa040c30ef80426e439f02a3f4e4137733cbaae1fec5.aotcache
-rwxr-xr-x 1 root root  104848 Jul 17 04:16 1bea8094b78a3910345d80af3d182390fda07ae5788352651eb7773505dc39af.aotcache
-rwxr-xr-x 1 root root  121772 Jul 17 04:16 9fb501baf5ceec1ff4c9abb0473492ab4a66893203b932e28dfaa5c73c05e191.aotcache
-rwxr-xr-x 1 root root  244608 Jul 17 04:16 b7695da977e205398b33922d61493cc1e436cd2321bb2e54fbdc39b95fabeff7.aotcache
-rwxr-xr-x 1 root root 2092260 Jul 17 04:16 fc4c52f3910ed57a088d19ab86c671358f5e917cd4e95b21fd08e4fd922c0aa2.aotcache

Even if rosettad is not running, it does not affect the operation of rosetta, so the existing functionality remains unchanged.

There is also a configuration to use Unix Domain Sockets instead of Abstract Sockets. However, it appears that some preparation is required on the VM guest side before mounting, and I was unable to make it work. Therefore, I am using the configuration with Abstract Sockets.

SeeAlso: https://developer.apple.com/documentation/virtualization/running_intel_binaries_in_linux_vms_with_rosetta#4239539

…abstract socket

This change allows starting the `/mnt/lima-rosetta/rosttad daemon` in the VM.
Without this change, it cannot be started as shown below:
```console
$ sudo /mnt/lima-rosetta/rosettad daemon
rosetta error: Failed bind socket: 98
Trace/breakpoint trap
```

Since `rosettad` needs to run in the same namespace as `rosetta` and must be executed directly from `/mnt/lima-rosetta/rosettad` (copies of `rosettad` do not work), it is difficult to use during container image builds or similar processes.

In a container, `rosettad` can be run as follows:
```console
$ docker run -it --rm --platform linux/amd64 --name test-rosettad -v/mnt/lima-rosetta:/mnt/lima-rosetta ubuntu
root@45f887cb0eaa:/# ls -la ~/.cache/rosettad
/usr/bin/ls: cannot access '/root/.cache/rosettad': No such file or directory
root@45f887cb0eaa:/# /mnt/lima-rosetta/rosettad daemon&
[1] 10
root@45f887cb0eaa:/# ls -la ~/.cache/rosettad
total 2700
drwxr-xr-x 2 root root    4096 Jul 17 04:16 .
drwxr-xr-x 3 root root    4096 Jul 17 04:16 ..
-rwxr-xr-x 1 root root  186600 Jul 17 04:16 169a47096736cdeb5714fa040c30ef80426e439f02a3f4e4137733cbaae1fec5.aotcache
-rwxr-xr-x 1 root root  104848 Jul 17 04:16 1bea8094b78a3910345d80af3d182390fda07ae5788352651eb7773505dc39af.aotcache
-rwxr-xr-x 1 root root  121772 Jul 17 04:16 9fb501baf5ceec1ff4c9abb0473492ab4a66893203b932e28dfaa5c73c05e191.aotcache
-rwxr-xr-x 1 root root  244608 Jul 17 04:16 b7695da977e205398b33922d61493cc1e436cd2321bb2e54fbdc39b95fabeff7.aotcache
-rwxr-xr-x 1 root root 2092260 Jul 17 04:16 fc4c52f3910ed57a088d19ab86c671358f5e917cd4e95b21fd08e4fd922c0aa2.aotcache
```

Even if `rosettad` is not running, it does not affect the operation of `rosetta`, so the existing functionality remains unchanged.

There is also a configuration to use Unix Domain Sockets instead of Abstract Sockets. However, it appears that some preparation is required on the VM guest side before mounting, and I was unable to make it work. Therefore, I am using the configuration with Abstract Sockets.

SeeAlso: https://developer.apple.com/documentation/virtualization/running_intel_binaries_in_linux_vms_with_rosetta#4239539

Signed-off-by: Norio Nomura <norio.nomura@gmail.com>
@norio-nomura norio-nomura force-pushed the set-rosetta-abstract-socket-caching-option branch from 644ad91 to 8df844e Compare July 18, 2024 04:16
Copy link
Member

@jandubois jandubois left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, the change looks fine to me.

I wonder how useful it is though to enable AOT compilation inside individual containers. Ideally you would want to share the rosettad cache between host and containers, but I wonder if that has potential race conditions.

Also the fact that you have to run rosettad inside each container makes it hard to use it with e.g. Kubernetes.

Anyways, tested successfully with Alpine and Ubuntu.

@jandubois jandubois added this to the v1.0 (tentative) milestone Jul 19, 2024
@jandubois jandubois merged commit 6fa9145 into lima-vm:master Jul 19, 2024
27 checks passed
@norio-nomura norio-nomura deleted the set-rosetta-abstract-socket-caching-option branch July 19, 2024 00:10
@norio-nomura
Copy link
Contributor Author

Thanks 🙏🏻

I feel we should add tests on macOS 14 to the CI, but there are already many tests that fail randomly, which makes me hesitant.

tmeijn pushed a commit to tmeijn/dotfiles that referenced this pull request Aug 21, 2024
This MR contains the following updates:

| Package | Update | Change |
|---|---|---|
| [lima-vm/lima](https://github.com/lima-vm/lima) | minor | `v0.22.0` -> `v0.23.1` |

MR created with the help of [el-capitano/tools/renovate-bot](https://gitlab.com/el-capitano/tools/renovate-bot).

**Proposed changes to behavior should be submitted there as MRs.**

---

### Release Notes

<details>
<summary>lima-vm/lima (lima-vm/lima)</summary>

### [`v0.23.1`](https://github.com/lima-vm/lima/releases/tag/v0.23.1)

[Compare Source](lima-vm/lima@v0.23.0...v0.23.1)

#### Changes

-   Fixed the CI to generate the release note ([#&#8203;2555](lima-vm/lima#2555))

#### Usage

```console
[macOS]$ limactl create
[macOS]$ limactl start
...
INFO[0029] READY. Run `lima` to open the shell.

[macOS]$ lima uname
Linux
```

***

The binaries were built automatically on GitHub Actions.
The build log is available for 90 days: https://github.com/lima-vm/lima/actions/runs/10441930092

The sha256sum of the SHA256SUMS file itself is `e93a48f3a011c25367da50ab3609bb28437fcde259371f005f8b234caa46efff` .

***

Release manager: [@&#8203;AkihiroSuda](https://github.com/AkihiroSuda)

### [`v0.23.0`](https://github.com/lima-vm/lima/releases/tag/v0.23.0)

[Compare Source](lima-vm/lima@v0.22.0...v0.23.0)

-   YAML:
    -   Add a `param` field for defining variables ([#&#8203;2498](lima-vm/lima#2498), thanks to [@&#8203;norio-nomura](https://github.com/norio-nomura))

-   vz:
    -   Prioritize rosetta over qemu-user-static ([#&#8203;2474](lima-vm/lima#2474), thanks to [@&#8203;norio-nomura](https://github.com/norio-nomura))
    -   Configura AOT caching options using an abstract socket ([#&#8203;2489](lima-vm/lima#2489), thanks to [@&#8203;norio-nomura](https://github.com/norio-nomura))

-   Templates:
    -   add `alpine-image` ([#&#8203;2360](lima-vm/lima#2360), thanks to [@&#8203;jandubois](https://github.com/jandubois))
    -   remove `centos-stream-8`, `deprecated/centos-7` ([#&#8203;2457](lima-vm/lima#2457))
    -   update to the latest revisions ([#&#8203;2553](lima-vm/lima#2553))

-   Governance:
    -   MAINTAINERS: invite Oleksandr Redko ([@&#8203;alexandear](https://github.com/alexandear)) as a Reviewer  ([#&#8203;2383](lima-vm/lima#2383))

Full changes: https://github.com/lima-vm/lima/milestone/46?closed=1
Thanks to [@&#8203;AdamKorcz](https://github.com/AdamKorcz) [@&#8203;AmedeeBulle](https://github.com/AmedeeBulle) [@&#8203;SmartManoj](https://github.com/SmartManoj) [@&#8203;afbjorklund](https://github.com/afbjorklund) [@&#8203;alexandear](https://github.com/alexandear) [@&#8203;danchr](https://github.com/danchr) [@&#8203;fwilhe2](https://github.com/fwilhe2) [@&#8203;jandubois](https://github.com/jandubois) [@&#8203;norio-nomura](https://github.com/norio-nomura) [@&#8203;tcooper](https://github.com/tcooper) [@&#8203;why168](https://github.com/why168)

</details>

---

### Configuration

📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined).

🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied.

♻ **Rebasing**: Whenever MR becomes conflicted, or you tick the rebase/retry checkbox.

🔕 **Ignore**: Close this MR and you won't be reminded about this update again.

---

 - [ ] <!-- rebase-check -->If you want to rebase/retry this MR, check this box

---

This MR has been generated by [Renovate Bot](https://github.com/renovatebot/renovate).
<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzNy40NDAuNyIsInVwZGF0ZWRJblZlciI6IjM3LjQ0MC43IiwidGFyZ2V0QnJhbmNoIjoibWFpbiIsImxhYmVscyI6WyJSZW5vdmF0ZSBCb3QiXX0=-->
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants