Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use cxx-rs for core.rs #2336

Merged
merged 5 commits into from
Dec 23, 2020
Merged
Show file tree
Hide file tree
Changes from 3 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .cci.jenkinsfile
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@ parallel rpms: {
shwrap("""
# fetch tags so `git describe` gives a nice NEVRA when building the RPM
git fetch origin --tags
ci/install-extra-builddeps.sh
ci/installdeps.sh
git submodule update --init

Expand Down
43 changes: 41 additions & 2 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 1 addition & 0 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@ serde_derive = "1.0.118"
serde_json = "1.0.60"
serde_yaml = "0.8.14"
libc = "0.2.81"
cxx = "1.0.18"
nix = "0.19.1"
glib-sys = "0.10.1"
glib = "0.10.3"
Expand Down
10 changes: 10 additions & 0 deletions Makefile-libpriv.am
Original file line number Diff line number Diff line change
Expand Up @@ -87,3 +87,13 @@ AM_V_GPERF_0 = @echo " GPERF " $@;
src/%.c: src/%.gperf Makefile
$(AM_V_at)$(MKDIR_P) $(dir $@)
$(AM_V_GPERF)$(GPERF) < $< > $@.tmp && mv $@.tmp $@

# Also we now use cxx.rs
rpmostree-cxxrs.h: rust/src/lib.rs
$(AM_V_GEN) cxxbridge rust/src/lib.rs --header > $@
rpmostree-cxxrs.cxx: rust/src/lib.rs
$(AM_V_GEN) cxxbridge --include rpmostree-cxxrs.h rust/src/lib.rs > $@
cxxrs_sources = rpmostree-cxxrs.h rpmostree-cxxrs.cxx
librpmostreepriv_sources += $(cxxrs_sources)
BUILT_SOURCES += $(cxxrs_sources)
GITIGNOREFILES += $(cxxrs_sources)
31 changes: 19 additions & 12 deletions Makefile-rpm-ostree.am
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,12 @@

bin_PROGRAMS += rpm-ostree

rpm_ostree_SOURCES = src/app/main.cxx \
rpm_ostree_SOURCES = src/app/main.cxx

noinst_LTLIBRARIES += librpmostreeinternals.la

librpmostreeinternals_la_SOURCES = \
src/app/libmain.cxx \
src/app/rpmostree-builtins.h \
src/app/rpmostree-db-builtins.h \
src/app/rpmostree-compose-builtins.h \
Expand Down Expand Up @@ -65,25 +70,30 @@ rpm_ostree_SOURCES = src/app/main.cxx \
$(NULL)

if BUILDOPT_ROJIG
rpm_ostree_SOURCES += \
librpmostreeinternals_la_SOURCES += \
src/app/rpmostree-ex-builtin-commit2rojig.cxx \
src/app/rpmostree-ex-builtin-rojig2commit.cxx \
src/app/rpmostree-compose-builtin-rojig.cxx \
$(NULL)
endif

nodist_rpm_ostree_SOURCES = $(dbus_built_sources) $(nodist_librpmostreepriv_sources)
nodist_librpmostreeinternals_la_SOURCES = $(dbus_built_sources) $(nodist_librpmostreepriv_sources)

rpmostree_common_cflags = -I$(srcdir)/src/app -I$(srcdir)/src/daemon \
-I$(srcdir)/src/lib -I$(srcdir)/src/libpriv -I$(libglnx_srcpath) \
-fvisibility=hidden \
-DG_LOG_DOMAIN=\"rpm-ostreed\" \
-DLIBDIR=\"$(libdir)\" -DPKGLIBDIR=\"$(pkglibdir)\" \
$(PKGDEP_RPMOSTREE_CFLAGS)
rpm_ostree_CFLAGS = $(AM_CFLAGS) $(rpmostree_common_cflags)
rpm_ostree_CXXFLAGS = $(AM_CXXFLAGS) $(rpmostree_common_cflags)
rpm_ostree_LDADD = $(PKGDEP_RPMOSTREE_LIBS) $(CAP_LIBS) libglnx.la librpmostree-1.la
$(PKGDEP_RPMOSTREE_CFLAGS) $(PKGDEP_RPMOSTREE_RS_CFLAGS)
rpmostree_bin_common_cflags = $(rpmostree_common_cflags)
rpmostree_bin_common_libs = $(PKGDEP_RPMOSTREE_LIBS) $(CAP_LIBS) libglnx.la librpmostree-1.la $(librpmostree_rust_path) $(PKGDEP_RPMOSTREE_RS_LIBS) -lstdc++
rpm_ostree_CFLAGS = $(AM_CFLAGS) $(rpmostree_bin_common_cflags)
rpm_ostree_CXXFLAGS = $(AM_CXXFLAGS) $(rpmostree_bin_common_cflags)
rpm_ostree_LDADD = librpmostreeinternals.la $(rpmostree_bin_common_libs)
EXTRA_rpm_ostree_DEPENDENCIES = libdnf.so.2
librpmostreeinternals_la_CFLAGS = $(AM_CFLAGS) $(rpmostree_common_cflags)
librpmostreeinternals_la_CXXFLAGS = $(AM_CXXFLAGS) $(rpmostree_common_cflags)
librpmostreeinternals_la_LIBADD = $(rpmostree_bin_common_libs)
EXTRA_librpmostreeinternals_la_DEPENDENCIES = libdnf.so.2

privdatadir=$(pkglibdir)
privdata_DATA = src/app/rpm-ostree-0-integration.conf
Expand All @@ -107,7 +117,7 @@ endif
endif
GITIGNOREFILES += tooling/target/ rpmostree-bindgen

librpmostree_rust_path = @abs_top_builddir@/target/@RUST_TARGET_SUBDIR@/librpmostree_rust.a
librpmostree_rust_path = $(top_srcdir)/target/@RUST_TARGET_SUBDIR@/librpmostree_rust.a
# If the target directory exists, and isn't owned by our uid, then
# we exit with a fatal error, since someone probably did `make && sudo make install`,
# and in this case cargo will download into ~/.root which we don't want.
Expand All @@ -133,9 +143,6 @@ endif
endif
BUILT_SOURCES += rpmostree-rust.h

rpm_ostree_CFLAGS += $(PKGDEP_RPMOSTREE_RS_CFLAGS)
rpm_ostree_LDADD += $(librpmostree_rust_path) $(PKGDEP_RPMOSTREE_RS_LIBS) -lstdc++

# Wraps `cargo test`. This is always a debug non-release build;
# the main thing here is we still drop the `target` dir in our build
# directory, since we nominally support srcdir != builddir.
Expand Down
65 changes: 32 additions & 33 deletions Makefile-tests.am
Original file line number Diff line number Diff line change
Expand Up @@ -20,39 +20,38 @@ endif

GITIGNOREFILES += ssh-config ansible-inventory.yml vmcheck-logs/ test-compose-logs/ tests/vmcheck/image.qcow2

# Disabled for now because we dropped librpmostreepriv.la
# testbin_cppflags = $(AM_CPPFLAGS) -I $(srcdir)/src/lib -I $(srcdir)/src/libpriv -I $(srcdir)/libglnx -I $(srcdir)/tests/common
# testbin_cflags = $(AM_CFLAGS) -fvisibility=hidden $(PKGDEP_RPMOSTREE_CFLAGS)
# testbin_ldadd = $(PKGDEP_RPMOSTREE_LIBS) librpmostree-1.la librpmostreepriv.la -lstdc++
#
# noinst_LTLIBRARIES += libtest.la
# libtest_la_SOURCES = tests/common/libtest.c
# libtest_la_CPPFLAGS = $(testbin_cppflags)
# libtest_la_CFLAGS = $(testbin_cflags)
# libtest_la_LIBADD = $(testbin_ldadd)
#
# tests_check_jsonutil_CPPFLAGS = $(testbin_cppflags)
# tests_check_jsonutil_CFLAGS = $(testbin_cflags)
# tests_check_jsonutil_LDADD = $(testbin_ldadd) libtest.la
#
# tests_check_postprocess_CPPFLAGS = $(testbin_cppflags)
# tests_check_postprocess_CFLAGS = $(testbin_cflags)
# tests_check_postprocess_LDADD = $(testbin_ldadd) libtest.la
#
# tests_check_test_utils_CPPFLAGS = $(testbin_cppflags)
# tests_check_test_utils_CFLAGS = $(testbin_cflags)
# tests_check_test_utils_LDADD = $(testbin_ldadd) libtest.la
#
# tests_check_test_sysusers_CPPFLAGS = $(testbin_cppflags)
# tests_check_test_sysusers_CFLAGS = $(testbin_cflags)
# tests_check_test_sysusers_LDADD = $(testbin_ldadd) libtest.la
#
# uninstalled_test_programs = \
# tests/check/jsonutil \
# tests/check/postprocess \
# tests/check/test-utils \
# tests/check/test-sysusers \
# $(NULL)
testbin_cppflags = $(AM_CPPFLAGS) -I $(srcdir)/src/lib -I $(srcdir)/src/libpriv -I $(srcdir)/libglnx -I $(srcdir)/tests/common
testbin_cflags = $(AM_CFLAGS) $(rpmostree_bin_common_cflags)
testbin_ldadd = librpmostreeinternals.la $(rpmostree_bin_common_libs)

noinst_LTLIBRARIES += libtest.la
libtest_la_SOURCES = tests/common/libtest.c
libtest_la_CPPFLAGS = $(testbin_cppflags)
libtest_la_CFLAGS = $(testbin_cflags)
libtest_la_LIBADD = $(testbin_ldadd)

tests_check_jsonutil_CPPFLAGS = $(testbin_cppflags)
tests_check_jsonutil_CFLAGS = $(testbin_cflags)
tests_check_jsonutil_LDADD = $(testbin_ldadd) libtest.la

tests_check_postprocess_CPPFLAGS = $(testbin_cppflags)
tests_check_postprocess_CFLAGS = $(testbin_cflags)
tests_check_postprocess_LDADD = $(testbin_ldadd) libtest.la

tests_check_test_utils_CPPFLAGS = $(testbin_cppflags)
tests_check_test_utils_CFLAGS = $(testbin_cflags)
tests_check_test_utils_LDADD = $(testbin_ldadd) libtest.la

tests_check_test_sysusers_CPPFLAGS = $(testbin_cppflags)
tests_check_test_sysusers_CFLAGS = $(testbin_cflags)
tests_check_test_sysusers_LDADD = $(testbin_ldadd) libtest.la

uninstalled_test_programs = \
tests/check/jsonutil \
tests/check/postprocess \
tests/check/test-utils \
tests/check/test-sysusers \
$(NULL)

uninstalled_test_scripts = \
tests/check/test-lib-introspection.sh \
Expand Down
1 change: 1 addition & 0 deletions ci/build.sh
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ set -xeuo pipefail
dn=$(dirname $0)
. ${dn}/libbuild.sh

${dn}/install-extra-builddeps.sh
${dn}/installdeps.sh

# create an unprivileged user for testing
Expand Down
7 changes: 7 additions & 0 deletions ci/install-extra-builddeps.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
#!/usr/bin/bash
# Install build dependencies not in the cosa buildroot already
set -xeuo pipefail
if ! command -v cxxbridge; then
ver=$(cargo metadata --format-version 1 | jq -r '.packages[]|select(.name == "cxx").version')
cargo install --root=/usr cxxbridge-cmd --version "${ver}"
fi
2 changes: 1 addition & 1 deletion configure.ac
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,7 @@ CC_CHECK_FLAGS_APPEND([WARN_CFLAGS], [CFLAGS], [\
-Werror=parenthesis \
-Werror=undef \
-Werror=misleading-indentation \
-Werror=missing-include-dirs -Werror=aggregate-return \
-Werror=missing-include-dirs \
-Wstrict-aliasing=2 \
-Werror=unused-result \
])])
Expand Down
77 changes: 37 additions & 40 deletions rust/src/core.rs
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
pub use self::ffi::*;
use crate::ffiutil;
use anyhow::Result;
use ffiutil::ffi_view_openat_dir;
use openat_ext::OpenatDirExt;

/// Guard for running logic in a context with temporary /etc.
Expand All @@ -13,27 +14,27 @@ pub struct TempEtcGuard {
renamed_etc: bool,
}

impl TempEtcGuard {
/// Create a context with a temporary /etc, and return a guard to it.
pub fn undo_usretc(rootfs: openat::Dir) -> anyhow::Result<Self> {
let has_usretc = rootfs.exists("usr/etc")?;
if has_usretc {
// In general now, we place contents in /etc when running scripts
rootfs.local_rename("usr/etc", "etc")?;
// But leave a compat symlink, as we used to bind mount, so scripts
// could still use that too.
rootfs.symlink("usr/etc", "../etc")?;
}

let guard = Self {
rootfs,
renamed_etc: has_usretc,
};
Ok(guard)
pub fn prepare_tempetc_guard(rootfs: i32) -> Result<Box<TempEtcGuard>> {
let rootfs = ffi_view_openat_dir(rootfs);
let has_usretc = rootfs.exists("usr/etc")?;
let mut renamed_etc = false;
if has_usretc {
// In general now, we place contents in /etc when running scripts
rootfs.local_rename("usr/etc", "etc")?;
// But leave a compat symlink, as we used to bind mount, so scripts
// could still use that too.
rootfs.symlink("usr/etc", "../etc")?;
renamed_etc = true;
}
Ok(Box::new(TempEtcGuard {
rootfs,
renamed_etc,
}))
}

impl TempEtcGuard {
/// Remove the temporary /etc, and destroy the guard.
pub fn redo_usretc(self) -> anyhow::Result<()> {
pub fn undo(&self) -> anyhow::Result<()> {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

From an out-of-band chat: this seems to be a cxx limitation, not being able to consume the ownership of the object itself.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would be cool if we could we make the undo happen in Drop::drop so we don't have to explicitly call it. This matches some other guards we have in C which used autocleanups to "unguard".

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We could do that actually! But I explicitly chose not to because there's no way to check for errors in drop(). (This gets into the whole rationale behind things like https://docs.rs/drop_bomb/0.1.5/drop_bomb/ that we use in https://docs.rs/openat-ext/0.1.9/openat_ext/struct.FileWriter.html )

if self.renamed_etc {
/* Remove the symlink and swap back */
self.rootfs.remove_file("usr/etc")?;
Expand All @@ -43,28 +44,24 @@ impl TempEtcGuard {
}
}

mod ffi {
#[cfg(test)]
mod test {
use super::*;
use glib_sys::GError;

#[no_mangle]
pub extern "C" fn ror_tempetc_undo_usretc(
rootfs: libc::c_int,
gerror: *mut *mut GError,
) -> *mut TempEtcGuard {
let fd = ffiutil::ffi_view_openat_dir(rootfs);
let res = TempEtcGuard::undo_usretc(fd).map(Box::new);
ffiutil::ptr_glib_error(res, gerror)
}
use std::os::unix::prelude::*;

#[no_mangle]
pub extern "C" fn ror_tempetc_redo_usretc(
guard_ptr: *mut TempEtcGuard,
gerror: *mut *mut GError,
) -> libc::c_int {
assert!(!guard_ptr.is_null());
let guard = unsafe { Box::from_raw(guard_ptr) };
let res = guard.redo_usretc();
ffiutil::int_glib_error(res, gerror)
#[test]
fn basic() -> Result<()> {
let td = tempfile::tempdir()?;
let d = openat::Dir::open(td.path())?;
let g = super::prepare_tempetc_guard(d.as_raw_fd())?;
g.undo()?;
d.ensure_dir_all("usr/etc/foo", 0o755)?;
assert!(!d.exists("etc/foo")?);
let g = super::prepare_tempetc_guard(d.as_raw_fd())?;
assert!(d.exists("etc/foo")?);
g.undo()?;
assert!(!d.exists("etc")?);
assert!(d.exists("usr/etc/foo")?);
Ok(())
}
}
12 changes: 12 additions & 0 deletions rust/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -10,11 +10,23 @@
mod ffiutil;
mod includes;

#[cxx::bridge(namespace = "rpmostreecxx")]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't the namespace name be e.g. rpmostreers to reflect the fact that it's actually calling into Rust code?

Copy link
Member Author

@cgwalters cgwalters Dec 22, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah but see it's bidirectional, cxx supports Rust calling C++ too and I just submitted #2413 to demo that.
Concretely anything listed in extern "C++" is expected to be in the target namespace too.

That said it could totally use a better name.

  • ror:: following the ror_ naming for bindgen?
  • rpmostree:: - Unimaginative but obvious
  • Some made up name based on...a bridge or something? skytrain:: ?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OTOH we can also change the name later; there's not a lot of code using this right now. It'll just get slightly more expensive to do so as usage increases.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I looked at this a bit more. It's an interesting topic; partly comes down to whether we see the "cxxrs" part as separate from the C/C++ part. And the core question there is, do we stick everything in the C++ side into the same rpmostree namespace? That'd mean that it'd just be e.g. download_to_fd and not rpmostreecxx::download_to_fd() when called.

This also conceptually would lead towards dropping the C function prefixing, e.g. instead of rpmostree_syscore_bump_mtime() it'd be rpmostree::syscore_bump_mtime(). My feeling right now is that'd be way too much churn; the high level goal here is to sprint towards thinning out the C/C++ side into Rust, not to spend time rewriting into "modern C++". But there will likely be some of the latter.

So I think for now let's just keep rpmostreecxx:: (and not stick any C++ code into a new namespace) because it highlights the C++/Rust-bridged code in our C-in-C++ code which just works differently for now.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right OK, got it. WDYT about cxxrs::? Anyway, yeah we can revisit later.

Also, definitely agreed re. not trying to make too much use of namespaces outside of cxxrs.

mod ffi {
// core.rs
extern "Rust" {
type TempEtcGuard;

fn prepare_tempetc_guard(rootfs: i32) -> Result<Box<TempEtcGuard>>;
fn undo(self: &TempEtcGuard) -> Result<()>;
}
}

mod cliwrap;
pub use cliwrap::*;
mod composepost;
pub use self::composepost::*;
mod core;
use crate::core::*;
mod history;
pub use self::history::*;
mod journal;
Expand Down
Loading