Skip to content

Commit

Permalink
Set the encoding to UTF-8 when reading string with FromBytes
Browse files Browse the repository at this point in the history
Change the encoding to UTF-8 when reading string, and also fall back to single-byte character encoding for unsupported string encodings.

The issue discovered is that the string is being processed one byte at a time. However, in UTF-8 encoding, characters can be represented by two or more bytes, which might cause encoding issues.

For example the character á is encoded incorrectly.
  • Loading branch information
kokosha authored and hasenbanck committed Dec 21, 2024
1 parent 507675e commit dd253e5
Showing 1 changed file with 6 additions and 3 deletions.
9 changes: 6 additions & 3 deletions ragnarok_bytes/src/from_bytes/implement.rs
Original file line number Diff line number Diff line change
Expand Up @@ -65,16 +65,19 @@ impl<T: FromBytes, const SIZE: usize> FromBytes for [T; SIZE] {

impl FromBytes for String {
fn from_bytes<Meta>(byte_reader: &mut ByteReader<Meta>) -> ConversionResult<Self> {
let mut value = String::new();
let mut value = Vec::<u8>::new();

while let Ok(byte) = byte_reader.byte::<Self>() {
match byte {
0 => break,
byte => value.push(byte as char),
byte => value.push(byte),
}
}
let result = String::from_utf8(value)
.map_err(|error| error.into_bytes())
.unwrap_or_else(|error| error.iter().map(|byte| *byte as char).collect());

Ok(value)
Ok(result)
}
}

Expand Down

0 comments on commit dd253e5

Please sign in to comment.