confusion specific to JavaScript

[nikomatsakis]

Brief summary

Alan is accustomed to JavaScript promises. He has design patterns in mind that don't work in Rust; he also gets confused by specific things around Rust futures. What are they?

Optional details

• (Optional) Which character(s) would be the best fit and why?
  • Alan: the experienced "GC'd language" developer, new to Rust
  • Grace: the systems programming expert, new to Rust
  • Niklaus: new programmer from an unconventional background
  • Barbara: the experienced Rust developer

[nikomatsakis]

Possible example: when you call an async function in JS, it begins executing immediately, and executes concurrently, whereas Rust futures are lazy.

[yoshuawuyts]

Some other topics that might be worth covering here:

• The difference between Node.js streams and Rust's streaming interfaces. (Main points to cover I see here are: backpressure, defining "through streams", consuming "readable streams", piping streams together, and error propagation)
• JavaScript comes with a runtime included, while Rust does not. This means even absolute beginners need to learn what a runtime is, learn about the options, and then proceed to pick one before they write their first line of code.
• In Node.js tracing interfaces "just work" for the most part — before this was done by monkey-patching built-ins, but now using the async_hooks API. Rust on the contrary does not provide a stable tracing API out of the box, and does not allow for monkey-patching. This means the experience is different for both tracing providers and consumers.
• Node.js comes with HTTP/HTTPS built-in, Rust does not. There are multiple HTTP stacks defined in userland, with different tradeoffs and interop guarantees.
• Maybe a fun one, but validating JSON in Rust is surprisingly easy and efficient. This is a big part of what web apps do, and maybe a chance to write a story about something that may delight users coming from other languages.

[vladinator1000]

Hi y'all! ❤️ Coming from JS here are some things that made an impression to me as a beginner.

Runtimes

🔁 I didn't know how to imagine what a runtime until someone said it's like a glorified while loop and that the special macro re-writes my code into something else.

Errors

⛔️ Handling errors in Tide was weird, for example if I wanted to upload an audio file, I'd have to do some parsing and writing to the filesystem. Usually I'd slap a question mark at the end of each function that returns a result, but when I wanted to turn those into meaningful errors I got confused. I still don't know how to properly do error handling so I use dyn Error result types everywhere if many errors can happen in a function 😬. Although the compiler catches so many things that I rarely need a debugger so there's that.

References and ownership

These took the longest to figure out. My dad and I have been coding an audio player server for fun every Saturday (🥳) for a while and it tripped us up when we wanted to do ƒancy hot-reloading. It was very rewarding to learn that you can shoo off the borrow-checker by lighting a candle and burning some incence in front of a portrait of Esteban, or just using .clone() everywhere. Oh, and passing closures around is really hard.

Love these

• Serialization and deserialization in Rust is so beautiful, I never wanna deal with parsing strings again.
• The .await at the end is super ergonomic, I love it so much
• Not having to wrap stuff in try/catch is excellent

[broccolihighkicks]

Some things that I would like to be improved in Rust, that I find useful in JS:

• Combinators

  • Promise.all, Promise.join, Promise.allSettled
  • Using use futures::join; join!(a(), b(), c()), my IDE (IntelliJ) has no idea what is inside of the macro brackets so there is no instant type checking.

• Pin'ing promises

  • Why do I need to pin a promise?
  • How can I store an array of promises and move them around?

• Async closures

  • Can I define a closure that is async?

• How do promises interact with threads?

  • In JS, I am used to relying on "when my code is running, it is the only thing mutating state".
    • With Rust, it requires a bit of research to understand if/when promises move to other threads. The compiler ensures a single writer, but I do not want to add extra cross-thread locking mechanisms for things that can be on the same thread.
    • Sometimes I want to move expensive work to a background thread and then return the result to a main event loop.

• Async iterables (for await of)

  • https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Statements/for-await...of
  • This is very useful for creating iterators over paginated API results.

• A standard runtime that will be supported long into the future.

  • I often get the impression I am building my code on runtime libraries that will change in a few months. I want my code to last forever.

[jlkiri]

There's a subtle difference in JS between returning Promises or awaiting them before returning. Suppose createPromise creates some Promise:

async function returnSomething() {
  return createPromise();
}

async function returnSomething() {
  return await createPromise();
}

The two snippets lead to different results. This article explains the difference in detail. It can be very confusing even for experienced JS devs. Rust works differently, which can bring even more confusion. For example, the code below panics because I don't .await return_number() inside return_call_async function.

use futures::executor::block_on;

async fn return_async_1() -> usize {
    return 1;
}

async fn return_call_async() -> usize {
    // return return_number().await;
    return return_number();
}

fn main() {
    let s = block_on(return_call_async());
    print!("{}", &s);
}

The provided error message is very uninformative and could be improved (e.g. explain why I can't do it this way? Is it possible by returning some other type? Should I return impl Future or something like that? Is returning usize from return_call_async correct? ).

error[E0308]: mismatched types
 --> src/main.rs:8:12
  |
3 | async fn return_number() -> usize {
  |                             ----- the `Output` of this `async fn`'s found opaque type
...
8 |     return return_number();
  |            ^^^^^^^^^^^^^^^ expected `usize`, found opaque type
  |
  = note:     expected type `usize`
          found opaque type `impl futures::Future`
help: consider `await`ing on the `Future`
  |
8 |     return return_number().await;
  |                           ^^^^^^

error: aborting due to previous error

[laurieontech]

When learning Rust a lot of emphasis is, rightly, placed on ownership/references and mutability. However, there are some marked differences in data types that are easily missed.

The various integer types/sizes is one.

The other is vectors versus arrays. It's all too easy to assume arrays are what you want because of the JS name similarity.

[evan-brass]

Cancellation:

In JavaScript, cancellation is handled with AbortControllers + AbortSignals. In Rust it is handled by not calling poll and the Drop trait. In JavaScript some APIs don't take an abort signal and you end up needing to wrap them: https://gist.github.com/jakearchibald/070c108c65e6db14db43d90d1c3a0305 In Rust, all Futures can be cancelled. In JavaScript cancellation is propagated via exception from bottom to top (by bubbling exception) instead of being propagated from top to bottom (by futures dropping their children) as it is in Rust.

Maybe it would help JavaScript users to have examples that show converting cancellation patterns in JS into Rust.

async function do_the_thing(data, signal) {
    const thing = create_a_thing(data);
    try {
        const res = await use_thing(thing, signal);
        return res + 2;
    } finally {
        thing.cleanup_the_thing();
    }
}

[jbr]

I think a lot of people coming from js look for Promise.all/Promise.allSettled/Bluebird.map(…, { concurrency: …}). It took me quite a while to realize that I needed to make a Vec<Future> into a stream, and even longer to find FuturesUnordered/BufferUnordered, and even then they're more awkward than I expected. The join/select macros are a confusing distraction for a lot of people, myself included.

Here was my thrashing attempt at understanding how to make outbound http requests with bounded concurrency: https://twitter.com/jacobrothstein/status/1212892911546683394, and critically for this wg, I ended up giving up on async and instead used rayon for this project: https://twitter.com/jacobrothstein/status/1213248527079313414

A little analysis: The futures crate has a lot of good ideas, but also some less-than-useful ones, and very few of them are documented in a didactic/tutorial manner, and the naming could stand to be improved for discoverability. There's no "so you want to iterate over this Vec<Future>/IntoIter<Item=Future>> in the order they resolve? Try FuturesUnordered. Want to bound the concurrency of that task? Try BufferUnordered. Want to do some stuff with streams? You'll need to pull in StreamExt. You don't need to do this with Iterator because it's in the std prelude."

Async-std and tokio have different approaches than those in futures, and it's not obvious why/when you need to reach for the futures crate (or, even more confusingly, futures-util / futures-lite, which also have differences). It's especially not obvious that a vec of futures needs to be turned into a stream in order to do async stuff with it, since in javascript there are lots of apis that operate on an array of promises without making it into a special thing. I'm much more familiar with async-std than tokio, but I don't believe async-std currently has any equivalent for FuturesUnordered or BufferUnordered, which leads people to try other solutions that don't work as well, like trying to fold a vec of futures into one giant join-ed or race-ed future

[y21]

Some things that I was struggling to understand after working with JavaScript for a long time:

• Sharing values: in JS, it's all a single thread, so sharing values just works. There's this smart event loop that makes it possible to have multiple "tasks" run concurrently on a single thread. No Mutex, RwLock, Arc etc. required like in async Rust. No need to worry about something not being Send or Sync.
• Laziness: Polling futures is required for something to happen. Calling an async function and not handling the returned future properly doesn't drive the future to completion. Nothing happens. async code in JavaScript just runs.
• Syntax: what's the difference between || async {} and async || {}? Why are async closures unstable, but moving the async keyword to the right side magically works?
• Pinning: Box<dyn Fn() -> Pin<Box<dyn Future<Output = ()>>>>?! Why is Pin needed when working with futures? What problem does it solve? Why do I need it, and sometimes I don't?
• Async traits: why can't my trait function be async?
• Blocking: Why is it fine and often preferred to use std::sync::Mutex over, say tokio::sync::Mutex? Blocking code should be avoided at all costs, so why is it fine to use std's Mutex?

[rhmoller]

This sums it up for me: https://github.com/rhmoller/blobs-and-bullets/blob/master/src/engine/image_future.rs

I just wanted to do the equivalent to this.

const loadImage = src => new Promise((resolve, reject) => {
  const img = new Image();
  img.onload = resolve;
  img.onerror = reject;
 img.src = src;
})

It took me forever to figure out. And now that I look at it a year later I can't remember how it works. (I haven't been writing Rust since)

[rhmoller]

BTW the above is not to dunk on Rust. I overall like the language and would like to pick it up again sometime.

[wraithan]

I recently bounced off rust async after thinking about using it for the 4th or 5th time over the years.

History

I'm somewhere between Alan and Barbara, but I remember before I was Alan and what pain I felt then. When Promises showed up in JS, I largely avoided them, hooked in via my well worn callbacks. This was rooted in a few concerns, some were performance reasons, but mostly it was interoperability with other code I had. In these days, using try/catch was mostly scoped around the very few things you knew could throw, to keep them from being uncaught and causing crashes, and most error handling happened via so called "error back" callbacks with the first argument being the potential error.

My early encounters with JS promises were learning a lot of interesting rules about how if you .then(thenFn, catchFn) it was different than .then(thenFn).catch(catchFn) and there was try/catch as a way to handle async errors, and now synchronous errors could be thrown and because they happened inside of a promise they would be considered an "uncaught rejection" and not ever bubble up to the user in Node.js until recent versions.

As I dove into that world of using Promises, I found there was a few different gotchas when converting callback code. Subtleties like how return await somePromise; is redundant in some cases and required to make a try/catch work as expected in others. Mix that in with various issues with where libraries would only work with some promise implementations, etc.

The ecosystem has come a long way in those years since Promises landed in Node.js, I now write Promise first code for 99% of my JS at work. If a library has a promise interaction I'll use it over a callback one for most things.

I've been a rustacean since just before 1.0, was lucky enough to attend RustCamp, and generally been hacking in rust for side projects for years and years now. I've written a lot of CLIs, a few art projects with my own rendering engine, and my own game engines. So I'm not coming at Rust fresh, I remember try!() and how often I used to have to specify lifetimes on everything. I've even taught local workshops and given talks about Rust.

Why I bounce off

I get vibes of the early JS promise when I start diving into Rust async. Also the amount I have to learn to start modifying my code confidently feels massive. Here's an ordered list, which is akin to some others on here

1. Runtimes. I understand core event loop dispatchers, I understand why they are needed. What I didn't grok in my last pass at this is things like async-std vs tokio vs others and how if I just arbitrarily pick one, how much pain am I setting myself up for? async-std has some features that talk about tokio support, libraries say they're for async-std specifically, does that mean I can't use them with libraries that are built for tokio or can I? I wanna hook my cart up to the horse that's going to win the race, increasing my chances of finding useful libraries and not having to convert all my code to a different ecosystem in a year because I bet wrong.
2. Libaries. Similar to the above, but different. Everything with async docs is all about "if you use this pile of async gear all together, everything is great!" but what if I don't have async libraries for everything? A common pattern I've seen (and implemented) is a pair of std::sync::mpsc::channel for a client of some sort that uses threads internally. How do I take common thread world patterns and convert them? Both in the slapdash equiv of throwing .unwrap() and hoping as well as the proper wrapper.
3. Are there dirt simple ways to add just a little async to a project? Again everything I see is all about "well I started out in this green field, then built up an async wonderland" but if I want to dip my toes in, in my own context where I want to just try out a Future in a spot where there is a thread doing some polling anyway. In JS I can have have piles of callbacks then decide "maybe instead of a callback for this one interaction, I'll try a promise" I can choose to not make my function async and use the .then().catch() interfaces mentioned above to keep from "infecting" the rest of the project.

My process that leads me to rust async and maybe part of why I bounce off:

1. Have some idea I want to code up, decide Rust is the way to go.
2. Code up a quick prototype using common things like structopt and serde.
3. Throw std::thread::spawn and std::sync::mpsc::channel at any connections I need to manage, such as TCP, serial port, my own custom hardware/protocol, etc.
4. Decide I want to orchestrate interactions between the sending and receiving better in my client layer. From JS experience this says rely on async/await.
5. Find the Rust Async book, spend most of a night's hack session reading that and poking around at projects to try to grok what all the terms mean these days, see who relies on what, etc.
6. Arbitrarily pick a runtime, this last time was async-std. Note I actually ran into a compile cache issue when I first added it to my deps, causing the linker to throw a bunch of errors until I did a cargo clean and tried again. Was unable to reproduce it, can't say it is "rust async" related, just a bump that took 20 minutes that could have been used on making this side project move foward.
7. Notice that serialport has two async implementations, one is tokio based, the other is mio based which maybe means out the gate I already made a bad choice with async-std since it isn't either of those. Spend another night reading these async wrappers to try to understand how they're laid out and why.
8. Decide that after 4 side project hack nights I was making zero progress on my project, and just added a try_recv based loop, did a little polling, and went on with my life having a working system, a little sad I still don't grok rust async and figure maybe I'll have to make a specific project to learn it sometime in the next few months... but maybe after the ecosystem has settled more.

I wrote this as a stream of consciousness so please forgive some transposed words and weird phrasing where my brain jumped about. I would like to have a grasp on rust async, maybe someday I'll have picked up enough tangentially and the ecosystem will come far enough that it does't seem like a huge lift to just try it out in a project.

Edit:
Actually, now that I think more on this... I can think of two more things that make it harder for me personally to move to Promises as well:

1. I learned about CSP, communicating sequential processes, early on as a programmer and it formed some of my fundamental opinions on how I think of dispatch of work between actors. This means using channels to synchronize threads is super native to me.
2. Rust taught me to not fear spawning threads. Straight up don't have to worry about it really. Just decide I want to split up some work, make some threads, toss as many channels as it needs in it for full duplex/simplex comms. Decide on how bounded the channels are for how synchronized the threads should be.

[geropl]

Yet another story:

I'm coming from a Java background with some familarity with C/C++, but am working with TypeScript/Node.js for >3 years now on a daily basis.

What really bugs me in Node.js is the thing that creating a Promise will automatically add it to the Event-Loop. This means that every time you do that (either explicitly with new Promise or implicitly with async function) and do not immediately either try...catch or .catch it, you basically created a UnhandledPromiseRejection landmine.

That's why I absolutely love Rust "explicit polling" approach. Coming from the problem mentioned above in Node.js lead me to always explicitly calling smol::Task::spawn whenever I want to execute something "in parallel".

Due to this approach I did not understand the recent fuzz about "Drop + Future considered harmful" at first:

• it's ok in smol to call smol::Task::spawn within smol::Task::spawn (as opposed to Tokio AFAIK)
• always using "spawn for parallel tasks + channels for output + Drop" does not lead to the problem in the first place

[FreddieGilbraith]

• I quite quickly understood that I'd need to bring my own async runtime, as apposed to JS where the async runtime is part of the language. But I still don't understand what I'm meant to do with multiple runtimes: there's tokio and async-std
  • Can code running in one runtime work with code running in another?
  • Do I have to pick a runtime at the start of my project and stick with it?
• The language around "polling" in the async docs really gave the wrong impression at first. I read it and thought "ok.... I thought polling was bad, but I guess the rust devs know what they're doing". It was quite a while before I discovered that it's not that kind of polling