Support for downloadable "soft fonts" (DRCS) #9164
Comments
j4james
added
the
Issue-Feature
ghost
added
Needs-Triage
|
This is extremely cool. I'd love to see it, and I'm sure @miniksa would agree. It does open up a minor can of worms though...
None of these concerns is significant enough to stop me from wanting it. Michael may feel differently? ¹ we may experience a 1-frame or (-1)-frame "tear" if the font changes and the sequence is passed through directly (using the quasi-broken conpty "pass through unknown sequences" feature, and without the renderer's consent/knowledge), right? but that's pretty minor. |
I'd very much like that too. And in theory it shouldn't be a problem to add support in the DX renderer, although it might be a bit tricky to test without the conpty side of things. I could probably hack something though. If you're also thinking about the double-width/height escape sequences, that's more of a political problem. I did initially implement a basic DX renderer for that, but it became clear that it would never work correctly with the way we're handling RTL in Windows Terminal, and I didn't want to start another argument on that subject.
I wouldn't say it definitely can't be done, but it's more complicated than you might think, because fonts can be updated after they've been rendered. To correctly handle that behaviour over conpty would probably require tracking character set data associated with each character. Maybe not impossible, but it seems pointlessly complicated for something that should be trivial with a pass-through version of conpty. |
|
It just occurred to me that it might be possible to get this working over conpty without that much extra effort if we limited our support to a single font buffer (which was standard for most of the earlier terminals). That way we probably only have to remember the last charset ID, and it should be safe to use that ID for forwarding all soft characters, regardless of the charset they were originally generated with. I may be missing something, though. I've been through this several times before where I thought I had a plan that would work, then later realized there was a complication I had overlooked. Either way, I'd still prefer to leave this for a follow-up PR. Just getting it working in conhost is going to be complicated enough as it is. |
DHowett
added
Product-Conhost
zadjii-msft
removed
the
Needs-Triage
Oh no. What if the conhost can handle all of these fonts and it just emits any of the complexity out the PTY side as sixels and we make Terminal understand sixels? :P
I do agree though that this is exciting and I'd love to see it. I'm not concerned about any of the issues up front to wholesale stop us from trying to do it.
If you're having trouble with the performance in GDI... I can look at it under the performance analyzer once there's a prototype and see if there's anything to be done. But generally speaking, BitBlt and StretchBlt ought to be hardware accelerated and pretty quick per https://docs.microsoft.com/en-us/windows-hardware/drivers/display/specifying-gdi-hardware-accelerated-rendering-operations. It might be faster to pre-stretch them all to the right size on another in-memory canvas and blt them over from there instead of making it do the stretch calculation every time.
And as for the DX side...I'm sure we can figure something out. I almost have the vaguest idea of writing all the glyphs into an in-memory font that exposes the interfaces and applying that... or a sprite map sort of thing with blits... hmmmm. I'm not that worried about the RTL either. If we have to refine RTL further, then we do.
I'm a bit worried about using the PUA and having someone have another legitimate use of it that gets in the way. But maybe it's OK to say you can't do both at the same time? I think the only way of avoiding it otherwise would be to attach a specific font override to pieces of the text buffer. Or some sort of flag. A bit ick, but possible.
NNNNNNNNNNNNNNNNNNNNNNNNNNNNNNN. Not excited about that potential. I want FON files to go die. But it might make it easier to do if there's some sort of bltting thing already established for this. |
Yeah, I am doing that now, and I think it was an improvement, but it still feels sluggish. My current plan is to see if I can get it working with a run-time raster font, and see if that's any faster. That way I think I can render a whole line of characters with a single GDI call instead of having to do a separate
This concerned me too. But if we only support a single font buffer, then it's only 96 characters, so I figured we should be able to find a safe gap somewhere that we can reserve. That said, I don't think adding a flag in the |
Yeah that's sort of the idea I was going for with what I was describing with DirectX... maybe making a fake in-memory font for it and letting those optimizations happen in the text drawing code. The problem with This is where things like |
|
https://devblogs.microsoft.com/oldnewthing/20170331-00/?p=95875 and https://docs.microsoft.com/en-us/windows/win32/api/wingdi/nf-wingdi-createdibsection? Maybe make a memory section with the bits, fill it up, share it with GDI, and push it all over at once? Woof. |
Now that you mention that, I did actually try using Also I need to slow down a bit and do thing properly. So far I've just been hacking things together to get a general idea of what works and what doesn't, but it's quite possible my hacky code is part of the problem. I'm not ruling anything out until I've had a chance to try and implement things properly. |
|
FYI, I've got a PR ready to submit for this, but it's branched off of PR #9307, so I'm waiting for that to merge first. This is still conhost only, and I've no immediate plans to work on the DX/conpty side of things, so there's no urgency to prioritize this from a Windows Terminal point of view. Just letting you know it's available if you want it. |
Yes I want it even if it's conhost only. At least there's a reference impl for me to work from to port it to DX eventually. Sorry I've been so out of it on reviewing these. |
|
Just a quick question to set expectations for this feature. @DHowett mentions that PCF fonts will never be supported but that DRCS fonts can be used in place of PCF. I googled conversions from PCF to DRCS but came up empty handed (it seems that DRCS font by itself does not yield many results). But how about
|
|
In theory, converting a raster .fnt file to DRCS shouldn't be too difficult, but there are a couple of limitations you need to be aware of.
But if you just want Cmatrix with the cool characters, there's already a fork that uses DRCS to achieve that. See the link and screenshot above. |
This PR introduces a mechanism via which DCS data strings can be passed through directly to the dispatch method that will be handling them, so the data can be processed as it is received, rather than being buffered in the state machine. This also simplifies the way string termination is handled, so it now more closely matches the behaviour of the original DEC terminals. * Initial support for DCS sequences was introduced in PR #6328. * Handling of DCS (and other) C1 controls was added in PR #7340. * This is a prerequisite for Sixel (#448) and Soft Font (#9164) support. The way this now works, a `DCS` sequence is dispatched as soon as the final character of the `VTID` is received. Based on that ID, the `OutputStateMachineEngine` should forward the call to the corresponding dispatch method, and its the responsibility of that method to return an appropriate handler function for the sequence. From then on, the `StateMachine` will pass on all of the remaining bytes in the data string to the handler function. When a data string is terminated (with `CAN`, `SUB`, or `ESC`), the `StateMachine` will pass on one final `ESC` character to let the handler know that the sequence is finished. The handler can also end a sequence prematurely by returning false, and then all remaining data bytes will be ignored. Note that once a `DCS` sequence has been dispatched, it's not possible to abort the data string. Both `CAN` and `SUB` are considered valid forms of termination, and an `ESC` doesn't necessarily have to be followed by a `\` for the string terminator. This is because the data string is typically processed as it's received. For example, when outputting a Sixel image, you wouldn't erase the parts that had already been displayed if the data string is terminated early. With this new way of handling the string termination, I was also able to simplify some of the `StateMachine` processing, and get rid of a few states that are no longer necessary. These changes don't apply to the `OSC` sequences, though, since we're more likely to want to match the XTerm behavior for those cases (which requires a valid `ST` control for the sequence to be accepted). ## Validation Steps Performed For the unit tests, I've had to make a few changes to some of the `OutputEngineTests` to account for the updated `StateMachine` processing. I've also added a new `StateMachineTest` to confirm that the data strings are correctly passed through to the string handler under all forms of termination. To test whether the framework is actually usable, I've been working on DRCS Soft Font support branched off of this PR, and haven't encountered any problems. To test the throughput speed, I also hacked together a basic Sixel parser, and that seemed to perform reasonably well. Closes #7316
ghost
added
Resolution-Fix-Committed
This PR adds conhost support for downloadable soft fonts - also known as dynamically redefinable character sets (DRCS) - using the `DECDLD` escape sequence. These fonts are typically designed to work on a specific terminal model, and each model tends to have a different character cell size. So in order to support as many models as possible, the code attempts to detect the original target size of the font, and then scale the glyphs to fit our current cell size. Once a font has been downloaded to the terminal, it can be designated in the same way you would a standard character set, using an `SCS` escape sequence. The identification string for the set is defined by the `DECDLD` sequence. Internally we map the characters in this set to code points `U+EF20` to `U+EF7F` in the Unicode private use are (PUA). Then in the renderer, any characters in that range are split off into separate runs, which get painted with a special font. The font itself is dynamically generated as an in-memory resource, constructed from the downloaded character bitmaps which have been scaled to the appropriate size. If no soft fonts are in use, then no mapping of the PUA code points will take place, so this shouldn't interfere with anyone using those code points for something else, as along as they aren't also trying to use soft fonts. I also tried to pick a PUA range that hadn't already been snatched up by Nerd Fonts, but if we do receive reports of a conflict, it's easy enough to change. ## Validation Steps Performed I added an adapter test that runs through a bunch of parameter variations for the `DECDLD` sequence, to make sure we're correctly detecting the font sizes for most of the known DEC terminal models. I've also tested manually on a wide range of existing fonts, of varying dimensions, and from multiple sources, and made sure they all worked reasonably well. Closes #9164
|
@j4james I assume you mean https://github.com/jhamby/cmatrix and you're absolutely right! I removed my apt version of However, using terminal Version: 1.9.1942.0 on Ubuntu 21.04 and the output with How did you run your tests for #10011? PS. Sorry for the very late reply |
|
@dotnetCarpenter FWIW #10011 hasn't shipped in any terminal release yet - 1.11 will be the first that has it |
|
However, Terminal doesn't support DRCS yet. Only the Windows Console host shipped inside Terminal does! |
|
@DHowett |
Description of the new feature/enhancement
Starting with the VT220 terminal, it was possible for apps to define their own "soft fonts", also known as dynamically replaceable character sets (DRCS). You would download the font to the terminal with a
DECDLDescape sequence, and assign it a character set ID that could then be designated via the usualSCSescape sequences.Some example use cases:
Custom fonts
More examples at https://vt100.net/dec/vt320/fonts
Simple monochromatic images
CMatrix with Japanese characters
Source: https://github.com/jhamby/cmatrix
Game sprites
Proposed technical implementation details (optional)
The screenshots above were taken from a POC I've been working on for conhost. In the current implemenation, when you've designated a DRCS character set, those characters get mapped to values in the Unicode PUA area. Then when the renderer encounters values in that range, it uses a
BitBltto render the glyph in place of the usualPolyTextOutcalls.The quality isn't fantastic - I'm just using
StretchBltto resize the provided glyphs to match the current font size. And the performance can seem a bit sluggish on apps like CMatrix when writing a lot of content to the screen. Nevertheless, I think it works reasonably well, and I'm sure someone smarter than me will have suggestions for how it could be improved.I'm still a long way from producing a PR, but I wanted to raise the issue now to see how much interest there was in the idea before spending too much time on it. Initially it would be conhost-only - I'm not sure about the conpty feasibility without #1173.
The text was updated successfully, but these errors were encountered: