Skip to content

Commit

Permalink
POC for step 1 of mozilla#419 - replace Handlemaps with Arcs.
Browse files Browse the repository at this point in the history
I played with this primarily to better understand how uniffi actually worked
and to try and get enough context to understand the challenges. It actually
worked out surprisingly well (although is nowhere near mergable)

The context for this is Ryan's [proposal for passing around object instances]( mozilla#419)
but I've only played with "step 1" - replacing Handlemaps with `Arc<>` or
`Arc<Mutex<>>`.

I kept support for both threadsafe and non-threadsafe interfaces, primarily
to avoid touching all the examples. It turns out the difference between
`[Threadsafe]` and not is quite trivial. This might change in the future
when we start supporting those other attributes, but maybe not. I'm still
on the fence about this.

There are some nasty hacks. The most obvious is using both `usize` and
`u64` for pointer values. `usize` is probably correct, but `u64`is still
handed around due to the lack of `usize` in Types. I tried using
`const *TypeName` but this meant that `TypeName` needed to be `pub`,
which isn't true in all the examples. You could argue that forcing
these types to be `pub` is actually the right thing, but I don't care
enough to actually do that :)

It's also incomplete. I'm currently failing to run the kotlin tests (they
fail in the same way in both this branch and on main - WSL FTW! :)

(I very lightly edited the code blocks below - removing comments and logging)

What it generates: in the non-threadsafe case, the constructor is:
```rust
pub extern "C" fn threadsafe_ffd6_Counter_new(
    err: &mut uniffi::deps::ffi_support::ExternError,
) -> usize /* *const Counter */ {
    match std::panic::catch_unwind(|| {
        let _new = std::sync::Mutex::new(Counter::new());
        let _arc = std::sync::Arc::new(_new);
        std::sync::Arc::into_raw(_arc)
    }) {
        Ok(ptr) => {
            *err = uniffi::deps::ffi_support::ExternError::default();
            ptr as usize
        }
        Err(e) => {
            *err = e.into();
            0 as usize /*`std::ptr::null()` is a compile error */
        }
    }
}
```

The threadsafe case is identical except for 1 line:
```rust
        let _new = ThreadsafeCounter::new();
```
(ie, no mutex)

The `free()` function is also quite trivial and identical in both cases
```rust
pub extern "C" fn ffi_threadsafe_ffd6_Counter_object_free(ptr: u64) {
    if let Err(e) = std::panic::catch_unwind(|| {
        assert_ne!(ptr, 0);
        // turn it into an Arc and let the value naturally drop.
        unsafe { std::sync::Arc::from_raw(ptr as *const Counter) };
    }) {
        uniffi::deps::log::error!("ffi_threadsafe_ffd6_Counter_object_free panicked: {:?}", e);
    }
}
```

The generated code for methods does have more variation between the 2, but not much. A non-threadsafe method:
```rust
pub extern "C" fn threadsafe_ffd6_Counter_busy_wait(
    ptr: u64,
    ms: i32,
    err: &mut uniffi::deps::ffi_support::ExternError,
) -> () {
    uniffi::deps::ffi_support::call_with_output(err, || {
        let _arc = unsafe { std::sync::Arc::from_raw(ptr as *const std::sync::Mutex<Counter>) };
        // This arc now "owns" the reference but we need an outstanding reference still.
        std::sync::Arc::into_raw(std::sync::Arc::clone(&_arc));
        let _retval = Counter::busy_wait(
            &mut *_arc.lock().unwrap(),
            <i32 as uniffi::ViaFfi>::try_lift(ms).unwrap(),
        );
        _retval
    })
}
```
for contrast, the thread-safe version:
```rust
pub extern "C" fn threadsafe_ffd6_ThreadsafeCounter_busy_wait(
    ptr: u64,
    ms: i32,
    err: &mut uniffi::deps::ffi_support::ExternError,
) -> () {
    uniffi::deps::ffi_support::call_with_output(err, || {
        let _arc = unsafe { std::sync::Arc::from_raw(ptr as *const ThreadsafeCounter) };
        // This arc now "owns" the reference but we need an outstanding reference still.
        std::sync::Arc::into_raw(std::sync::Arc::clone(&_arc));
        let _retval =
            ThreadsafeCounter::busy_wait(&*_arc, <i32 as uniffi::ViaFfi>::try_lift(ms).unwrap());
        _retval
    })
}

```
(As in the current version, there are small differences depending on whether an error is returned or not, but that's not particularly interesting here)

The "keep the Arc alive by leaking a clone" wasn't immediately obvious to me until Python kept crashing :)
  • Loading branch information
mhammond committed Mar 30, 2021
1 parent 67fcfaf commit ca510de
Show file tree
Hide file tree
Showing 5 changed files with 79 additions and 43 deletions.
1 change: 0 additions & 1 deletion uniffi/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,6 @@ pub mod deps {
pub use anyhow;
pub use bytes;
pub use ffi_support;
pub use lazy_static;
pub use log;
pub use static_assertions;
}
Expand Down
7 changes: 4 additions & 3 deletions uniffi_bindgen/src/interface/object.rs
Original file line number Diff line number Diff line change
Expand Up @@ -232,6 +232,7 @@ impl Constructor {
self.ffi_func.name.push_str("_");
self.ffi_func.name.push_str(&self.name);
self.ffi_func.arguments = self.arguments.iter().map(Into::into).collect();
// XXX - we need this to be some kind of pointer abstraction - `usize`?
self.ffi_func.return_type = Some(FFIType::UInt64);
Ok(())
}
Expand Down Expand Up @@ -283,8 +284,8 @@ impl APIConverter<Constructor> for weedle::interface::ConstructorInterfaceMember

// Represents an instance method for an object type.
//
// The in FFI, this will be a function whose first argument is a handle for an
// instance of the corresponding object type.
// The in FFI, this will be a function whose first argument is a pointer,
// represented as an integer, to an instance of the corresponding object type.
#[derive(Debug, Clone)]
pub struct Method {
pub(super) name: String,
Expand Down Expand Up @@ -314,7 +315,7 @@ impl Method {

pub fn first_argument(&self) -> Argument {
Argument {
name: "handle".to_string(),
name: "ptr".to_string(),
// TODO: ideally we'd get this via `ci.resolve_type_expression` so that it
// is contained in the proper `TypeUniverse`, but this works for now.
type_: Type::Object(self.object_name.clone()),
Expand Down
1 change: 1 addition & 0 deletions uniffi_bindgen/src/scaffolding.rs
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,7 @@ mod filters {
use super::*;
use std::fmt;

#[allow(dead_code)] // we should kill this if a need for it doesn't come back.
pub fn choose(
expr_value: &bool,
then_value: &dyn fmt::Display,
Expand Down
36 changes: 18 additions & 18 deletions uniffi_bindgen/src/templates/ObjectTemplate.rs
Original file line number Diff line number Diff line change
@@ -1,36 +1,36 @@
// For each Object definition, we assume the caller has provided an appropriately-shaped `struct`
// with an `impl` for each method on the object. We create a `ConcurrentHandleMap` for safely handing
// with an `impl` for each method on the object. We create an `Arc` in
// `[Threadsafe]` interfaces or `Arc<Mutex<>>` otherwise, for for "safely" handing
// out references to these structs to foreign language code, and we provide a `pub extern "C"` function
// corresponding to each method.
//
// Note that "safely" is in "scare quotes" - that's because we use functions on
// an `Arc` that are inherently unsafe, but the code we generate is safe in
// practice.
//
// If the caller's implementation of the struct does not match with the methods or types specified
// in the UDL, then the rust compiler will complain with a (hopefully at least somewhat helpful!)
// error message when processing this generated code.
{% let handle_map = format!("UNIFFI_HANDLE_MAP_{}", obj.name().to_uppercase()) -%}

uniffi::deps::lazy_static::lazy_static! {
{%- let handle_map_type = obj.threadsafe()|choose(
"uniffi::ffi::handle_maps::ArcHandleMap",
"uniffi::ffi::handle_maps::MutexHandleMap")
%}
#[doc(hidden)]
static ref {{ handle_map }}: {{ handle_map_type }}<{{ obj.name() }}>
= Default::default();
}

{% let ffi_free = obj.ffi_object_free() -%}
#[doc(hidden)]
#[no_mangle]
pub extern "C" fn {{ ffi_free.name() }}(handle: u64) {
let _ = {{ handle_map }}.delete_u64(handle);
{% let ffi_free = obj.ffi_object_free() -%}
#[doc(hidden)]
#[no_mangle]
pub extern "C" fn {{ ffi_free.name() }}(ptr: u64) {
if let Err(e) = std::panic::catch_unwind(|| {
assert_ne!(ptr, 0);
// turn it into an Arc and let the value naturally drop.
unsafe { std::sync::Arc::from_raw(ptr as *const {{ obj.name() }}) };
}) {
uniffi::deps::log::error!("{{ ffi_free.name() }} panicked: {:?}", e);
}
}

{%- for cons in obj.constructors() %}
#[allow(clippy::all)]
#[doc(hidden)]
#[no_mangle]
pub extern "C" fn {{ cons.ffi_func().name() }}(
{%- call rs::arg_list_ffi_decl(cons.ffi_func()) %}) -> u64 {
{%- call rs::arg_list_ffi_decl(cons.ffi_func()) %}) -> usize /* *const {{ obj.name() }} */ {
uniffi::deps::log::debug!("{{ cons.ffi_func().name() }}");
// If the constructor does not have the same signature as declared in the UDL, then
// this attempt to call it will fail with a (somewhat) helpful compiler error.
Expand Down
77 changes: 56 additions & 21 deletions uniffi_bindgen/src/templates/macros.rs
Original file line number Diff line number Diff line change
Expand Up @@ -6,9 +6,14 @@
{{ func.name() }}({% call _arg_list_rs_call(func) -%})
{%- endmacro -%}

{%- macro to_rs_call_with_prefix(prefix, func) -%}
{%- macro to_rs_call_with_prefix(arg_name, func, obj) -%}
{{ func.name() }}(
{{- prefix }}{% if func.arguments().len() > 0 %}, {% call _arg_list_rs_call(func) -%}{% endif -%}
{% if obj.threadsafe() %}
&*{{- arg_name }}
{% else %}
&mut *{{- arg_name }}.lock().unwrap()
{% endif %}
{% if func.arguments().len() > 0 %}, {% call _arg_list_rs_call(func) -%}{% endif -%}
)
{%- endmacro -%}

Expand Down Expand Up @@ -47,33 +52,63 @@
{% macro to_rs_constructor_call(obj, cons) %}
{% match cons.throws() %}
{% when Some with (e) %}
UNIFFI_HANDLE_MAP_{{ obj.name()|upper }}.insert_with_result(err, || -> Result<{{obj.name()}}, {{e}}> {
let _retval = {{ obj.name() }}::{% call to_rs_call(cons) %}?;
Ok(_retval)
})
todo!("XXX - catch_unwind like below...");
let constructed = {{ obj.name() }}::{% call to_rs_call(cons) %}?;
let arc = std::sync::Arc::new(constructed);
Ok(std::sync:Arc::into_raw(arc))
{% else %}
UNIFFI_HANDLE_MAP_{{ obj.name()|upper }}.insert_with_output(err, || {
{{ obj.name() }}::{% call to_rs_call(cons) %}
})
match std::panic::catch_unwind(|| {
{%- if obj.threadsafe() %}
let _new = {{ obj.name() }}::{% call to_rs_call(cons) %};
{%- else %}
let _new = std::sync::Mutex::new({{ obj.name() }}::{% call to_rs_call(cons) %});
{%- endif %}
let _arc = std::sync::Arc::new(_new);
std::sync::Arc::into_raw(_arc)
}) {
Ok(ptr) => {
*err = uniffi::deps::ffi_support::ExternError::default();
ptr as usize
},
Err(e) => {
*err = e.into();
0 as usize /*`std::ptr::null()` is a compile error */
}
}
{% endmatch %}
{% endmacro %}

{% macro get_arc(obj) -%}
{% if obj.threadsafe() %}
let _arc = unsafe { std::sync::Arc::from_raw(ptr as *const {{ obj.name() }}) };
{% else %}
let _arc = unsafe { std::sync::Arc::from_raw(ptr as *const std::sync::Mutex<{{ obj.name() }}>) };
{% endif %}
// This arc now "owns" the reference but we need an outstanding reference still.
std::sync::Arc::into_raw(std::sync::Arc::clone(&_arc));
{% endmacro %}


{% macro to_rs_method_call(obj, meth) -%}
{% let this_handle_map = format!("UNIFFI_HANDLE_MAP_{}", obj.name().to_uppercase()) -%}
{% if !obj.threadsafe() -%}
use uniffi::UniffiMethodCall;
{%- endif -%}
{% match meth.throws() -%}
{% when Some with (e) -%}
{{ this_handle_map }}.method_call_with_result(err, {{ meth.first_argument().name() }}, |obj| -> Result<{% call return_type_func(meth) %}, {{e}}> {
let _retval = {{ obj.name() }}::{%- call to_rs_call_with_prefix("obj", meth) -%}?;
Ok({% call ret(meth) %})
})
uniffi::deps::ffi_support::call_with_result(
err,
|| -> Result<_, {{ e }}> {
{% call get_arc(obj) %}
let _retval = {{ obj.name() }}::{%- call to_rs_call_with_prefix("_arc", meth, obj) -%}?;
Ok({% call ret(meth) %})
},
)
{% else -%}
{{ this_handle_map }}.method_call_with_output(err, {{ meth.first_argument().name() }}, |obj| {
let _retval = {{ obj.name() }}::{%- call to_rs_call_with_prefix("obj", meth) -%};
{% call ret(meth) %}
})
uniffi::deps::ffi_support::call_with_output(
err,
|| {
{% call get_arc(obj) %}
let _retval = {{ obj.name() }}::{%- call to_rs_call_with_prefix("_arc", meth, obj) -%};
{% call ret(meth) %}
},
)
{% endmatch -%}
{% endmacro -%}

Expand Down

0 comments on commit ca510de

Please sign in to comment.