Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

9x/ME: Implement proper handling for console writes where character lengths differ between utf16 and target codepage #14

Open
seritools opened this issue Jan 1, 2024 · 0 comments
Labels
enhancement New feature or request help wanted Extra attention is needed

Comments

@seritools
Copy link
Member

seritools commented Jan 1, 2024

For #13 I have implemented a kind of crappy workaround. This ticket tracks actually implementing a proper solution.

The workaround:

if !compat::is_windows_nt() {
// FIXME: This function should manually convert to the target codepage on 9x/ME, and
// handle incomplete writes by calculating how many utf8-effective bytes were written.
return Ok(utf8.len());
}

Problem description

Whenever the "character count" mismatches between the UTF-16 side and the codepage size, the "amount of characters written" also mismatches between both sides.

Solution idea

Roguh idea for the conversion code

Becasue of the multiple, lossy conversions needed on 9x/Me (subset of UTF8 → WTF16 → codepage), the only sensible way would be to do the code page conversion in the stdlib code, then loop until the entire converted buffer has been written, thus confirming that the entire input buffer of up to MAX_BUFFER_SIZE bytes have been written. Only that way we can ensure that all input characters are accounted for in some capacity.

@seritools seritools added enhancement New feature or request help wanted Extra attention is needed labels Jan 1, 2024
@seritools seritools changed the title Implement proper MBCS handling for console writes 9x/ME: Implement proper MBCS handling for console writes Jan 1, 2024
@seritools seritools changed the title 9x/ME: Implement proper MBCS handling for console writes 9x/ME: Implement proper handling for console writes where character lengths differ between utf16 and target codepage Jan 1, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request help wanted Extra attention is needed
Projects
None yet
Development

No branches or pull requests

1 participant