Why this cannot borrow multipart as mutable more than once at a time error?

[frederikhors]

I have this code:

pub async fn upload(
    mut multipart: Multipart,
) -> Result<Response, StatusCode> {
    while let Some(field) = multipart.next_field().await.unwrap() {
        let filename = if let Some(filename) = field.file_name() {
            filename.to_string()
        } else {
            continue;
        };

        let body_with_io_error = field.map_err(|err| io::Error::new(io::ErrorKind::Other, err));

        let body_reader = StreamReader::new(body_with_io_error);

        // futures::pin_mut!(body_reader);

        let body_reader = Box::pin(body_reader);

        let res = client.upload(filename, body_reader).await?;

        return Ok(Response::builder().status(StatusCode::CREATED).unwrap());
    }

    Err(StatusCode::INTERNAL_SERVER_ERROR)
}

but the client.upload() signature is:

pub async fn upload(filename: &str, reader: Pin<Box<(dyn AsyncRead + Send + Sync)>>) -> Result<(), ()>;

It error with:

error[E0499]: cannot borrow `multipart` as mutable more than once at a time
   |
29 |     while let Some(field) = multipart.next_field().await.unwrap() {
   |                             ^^^^^^^^^^^^^^^^^^^^^^ `multipart` was mutably borrowed here in the previous iteration of the loop
...
60 |                 body_reader,
   |                 ----------- cast requires that `multipart` is borrowed for `'static`

How can I fix this?

[davidpdrsn]

The signature of next_field is

pub async fn next_field(&mut self) -> Result<Option<Field<'_>>, MultipartError>

The <'_> in Field<'_> means it contains a reference to the &mut self in this case. Think of it as Field<'multipart> where 'multipart is the lifetime of multipart.

We do this because you can only consume one field at a time. Field is sorta represents a "pointer" into the streaming request body and you cannot consume the stream from multiple places at once. axum enforces this statically whereas multer (the library axum uses under the hood) checks it at runtime https://docs.rs/multer/latest/multer/struct.Multipart.html#field-exclusivity.

Next Box<dyn T> has an implicit 'static so it is really Box<dyn T + 'static>. That means T cannot contain references, so it can't be like Vec<&'a Foo> or Field<'multipart>. So Pin<Box<(dyn AsyncRead + Send + Sync)>> is Pin<Box<(dyn AsyncRead + Send + Sync + 'static)>>. That doesn't work because Field<'_> isn't 'static because it contains a reference to multipart.

I'm not sure why the library uses that signature, could just be an oversight or maybe there is some technical reason for it. I'm honestly not sure if there is a way to resolve it except for buffering the field bytes in memory with https://docs.rs/axum/latest/axum/extract/multipart/struct.Field.html#method.bytes but I'm assuming you want to avoid that.

[frederikhors]

@davidpdrsn I'm trying avoiding multipart and it works amazingly well! I'm happy!

Just one thing: how can I get the filename now? The filename before was in multipart field "name".

I was using this code:

while let Some(field) = multipart.next_field().await.unwrap() {
  let filename = if let Some(filename) = field.file_name() {
      filename.to_string()
  } else {
      continue;
  };
// ...
}

Now I'm using this code:

pub async fn upload(
    stream: BodyStream,
    TypedHeader(content_length): TypedHeader<ContentLength>,
) -> Result<Response, StatusCode> {
    let stream = stream.map_err(|err| io::Error::new(io::ErrorKind::Other, err));

    let created_file = file_create.execute(
            "filename".to_string(),
            content_length.0,
            Box::pin(StreamReader::new(stream)),
        )
        .await?;

    Ok(Response::builder().status(StatusCode::CREATED).unwrap())
}

What to use now?

[davidpdrsn]

You could send that in a query parameter instead maybe.

[frederikhors]

The thing is they are using a javascript client that sends multipart form. I cannot decide for that side... Is there a way to detect the filename?

[davidpdrsn]

The thing is they are using a javascript client that sends multipart form

Well in that case you can't just switch to BodyStream. I didn't realize you didn't control the client.

[praveenperera]

@frederikhors @davidpdrsn was there a solution to this, i'm also trying to stream a field from multipart to a reqwest body for upload.

    // find the field and stream upload to GCS
    while let Some(field) = multipart.next_field().await.wrap_err("cant get field")? {
        let name = field.name().unwrap_or_default().to_string();

        if name != "file" {
            continue;
        }

        let body = Body::wrap_stream(field);

        client
            .upload_multiple_chunk(body, &chunk_size)
            .await
            .wrap_err("error uploading chunk")?;

        return Ok((StatusCode::OK, "ok"));
    }

I get the same error on multipart needing to be static, upload_multiple_chunk expects Into<Body>