RFX : Support large screens

[matt335672]

Fixes #2068

Many thanks to @trishume for supplying the greater part of this code.

There are two commits in this code. The first makes the max size for a fastpath surface update PDU potentially much bigger if the client has a large display. This approach is taken by freerdp, and has already been validated as part of #2068. An upper limit is placed on this value to prevent runaway memory allocations.

The second is a patch supplied by @trishume. This patch also depends on a change to the librfxcodec repository, as the interface between the two packages has altered. I've tested this patch on my own setup by setting the max size for the fastpath update PDU to a small value (1MB).

[metalefty]

This patch also depends on a change to the librfxcodec repository, as the interface between the two packages has altered.

So the version of librfxcodec and xrdp must sync?

[matt335672]

Yes - the package versions need to sync.

The way this works (if my understanding of git submodules is correct) is that the change gets made to librfxcodec, and then the librfxcodec commit is added in to the main project by making a commit in that repository.

[matt335672]

I found a small memory over-allocation in the RFX encoder setup, but this is now also fixed. I've also merged in a small change to librfxcodec which calculates the maximum tile size in a more explicit manner.

Barring the as-yet-unduplicated problem that @RolKau has reported in #2068 (which we can look at later) I think this is ready for v0.9.18. More testing still welcome.

[Nexarian]

On comment -- You've included limits.h in some of the files, yet still hard code numbers like 1024 and 16384 -- I understand those numbers aren't in limits.h, but they still feel pretty standard to me. I wonder if we should create an "XRDP_LIMITS.H" file or some sort that has all of these included so we don't need to continue to be so verbose.

[matt335672]

Thanks.

Explicit constants from MS specs (e.g. [MS-RPBCGR]) should be in an include file which has the name name as the spec. There's also xrdp_constants.h where we can put things which don't belong there, if we feel there's a need.

Are you referring to the 16384 in this line?

result = x_tiles * y_tiles * 16384;

I can be more explicit about this value if you like. It's taken from this snippet in FreeRDP:-

https://github.com/FreeRDP/FreeRDP/blob/98348ef62aff75a26ee2824b96a8a196de63f273/libfreerdp/core/capabilities.c#L2495-L2510

essentially it's allowing 32 bits for each pixel of a 64x64 tile.