Deserializing a flattened struct containing a ByteBuf fails to deserialize with invalid UTF-8.

[scrblue]

As per the comment for deserialize_bytes, it is expected that you can deserialize a non-UTF-8 string into a ByteBuf without failure. However, with flattened structures, deserialize_map is called in place of deserialize_struct (see serde-rs/serde#1529), meaning members are deserialized with deserialize_any bypassing deserialize_bytes. deserialize_any assumes values surrounded by quotation marks are valid UTF-8 strings and returns an error otherwise.

An example follows

use serde::{Deserialize, Serialize};

fn main() {
    let a_success: A = serde_json::from_slice(b"{\"b\": {\"buf\": \"\xe5\x00\xe5\"}}").unwrap();
    println!("A Success: {:?}", a_success);

    let a_flat_success: AFlat = serde_json::from_slice(b"{\"buf\": \"abc\"}").unwrap();
    println!("A Flat Success: {:?}", a_flat_success);
    
    let a_flat_failure: AFlat = serde_json::from_slice(b"{\"buf\": \"\xe5\x00\xe5\"}").unwrap();
}

#[derive(Debug, Deserialize, Serialize)]
struct A {
    b: B,
}

#[derive(Debug, Deserialize, Serialize)]
struct AFlat {
    #[serde(flatten)]
    b: B,
}

#[derive(Debug, Deserialize, Serialize)]
struct B {
    buf: serde_bytes::ByteBuf,
}

[scrblue]

As reported in the issue I just mentioned this one in, deserialize_any is also used on all enums with tag directives meaning this deviation from expected results appears in multiple locations.

The fix you can make here would just be to ensure the deserialize_any function matches the functionality of the other deserialize_... functions. My PR to this crate will fix it in the case of deserializing into ByteBufs, but I'm not sure if other areas are affected as no other eccentricities have appeared in my use of serde and serde_json.

To fix this in Serde would likely require more fundamental changes that I don't have the expertise to make a PR for.

[jonasbb]

Since the issue mentions flattening and deserialize_any, the most likely culprit here is serde-rs/serde#1183.

[lucacasonato]

Also relevant is #890