Cross Platform: Add support for Anchor/Complementary Packages.

[migueldeicaza]

The above is just a working title, I am not attached to any names.

The short version is that we would like NuGet to support a notion of an anchor package as well as complementary packages linked to the anchor which augment the view of the core package when they are published. This is not a theoretical issue, this is currently preventing several of our libraries from supporting .NET Core, Linux/Unix, various existing gaming consoles, and secret platforms under development.

Let me explain.

One of the challenges that we have with some of our .NET libraries is that they need native code to support them.

Consider our high performance 2D graphics library for .NET, SkiaSharp. It is powered by Google Chrome’s C++ engine, and it is surfaced as a beautifully bound C# API. So we need to distribute both the native library, as well as the managed library for it.

The managed part is very easy to deal with, we can build it anywhere, and package it anywhere.

The native library is very hard, because there are no standard solutions for cross-compiling C/C++ code from any host to any target, so what we have to do is build the various native libraries on their host platforms. For Skia, today this matrix looks like this:

• MacOS libraries built on MacOS
• iOS, tvOS libraries built on MacOS
• Android libraries built on Windows
• UWP, WPF, Win32 libraries built on Windows

Then we write scripts to copy the files from one set of hosts to the other and put the files in the NuGet.

At Xamarin, our focus has been the Android and Apple platforms complementing Windows, so we have enough scripts and tools to help us do this that we are happy. But in a world where Linux plays a larger role, we would need to add to our setup each Linux distribution that we would support. This presents three problems for NuGet maintainers.

In the Linux world, binaries from one distribution can not in generally be shared with binaries from another distribution, even on the same platform. So we cannot just produce a Linux/x86 binary. We actually have to produce an Ubuntu/Linux/x86, a RHEL/Linux/x86, a Fedora/Linux/x89, a Debian/Linux/x86, a MinLinux/x86, a SUSE/Linux/x86, an Alpine/Linux/x86.

It is quite a mouthful. Let me add another layer of complexity. Not only do we have to produce different binaries per Linux distribution, often, we have to generate different binaries for major versions of the Linux distributions! So that innocent looking Debian/Linux/x86 practically, it means producing 3 versions of the binary. 2 for RHEL, likely 2 for SUSE, likely 3-4 for Ubuntu and so on.

The first problem is that it requires both more work on orchestration (either manual, or CI) to put together the NuGet package with all of the unmanaged assets that are required. There are logistical issues for someone that would like to generate all of those binaries as you can imagine.

The second problem is that it is not future proof. If a new operating system or platform needs to be supported, say a new Linux distribution comes out or .NET core starts supporting Solaris, it is in general not possible to publish just the unmanaged artifacts that are required. So either we under-serve the new platforms, or if we go through the motions of doing the work, we can create Solaris, and release a new package that people must upgrade to.

The third problem has to do with trade secrets/privacy. .NET is a popular technology in gaming, and it now runs on platforms that are under strict NDAs, this includes XboxOne, PlayStation, WiiU, Nintendo 3DS as well as other platforms that have not been disclosed to the public. It is not currently possible to release binaries for these platforms in the main NuGet package without breaking the confidentiality agreement. Users in this world are currently underserved.

My proposal is to implement in NuGet support for publishing “complementary” packages that are anchored against an original package. This would allow scenarios where the developer can publish an anchor package, for the sake of this argument, let us say that Xamarin just published “SkiaSharp” with support for MacOS, Android, iOS and Windows. And now, the .NET Core team works with Red Hat, and they create a binary for SkiaSharp for Fedora and RHEL for x86 and x86-64, and they submit the peer package, “SkiaSharp.Peer.RHEL(x86,x64)”. We then launch our new PlayStation support, and we publish “SkiaSharp.Peer.PlayStation4”.

During the development phases of a new gaming platform, we are able to internally create a NuGet feed with our packages, so we publish internally “SkiaSharp.Peer.XboxTwo”.

This would require the NuGet client to consider the package “SkiaSharp” to be the concatenation of SkiaSharp, plus all of its peers.

Now, there is a trust issue that needs to be solved. We do not want any untruted party to publish complementary SkiaSharp binaries, mostly for secureity reasons (trust issues, brand issues, prevent exploits). We could do this with self signed certificates, so as the original author of SkiaSharp, I could grant other trusted parties to publish packages without disclosing my key. So Rich would share with me his team’s certificate, and we would trust it, and then he could publish SkiaSharp binaries for all of the platforms the .NET team is supporting without Xamarin’s intervention.

End users would need a way of forcing trust, for example, Nintendo would not likely disclose the existence of a NextGen Wii to the public, or Apple disclose the existence of the AppleCar to the world, yet, they should be able to say “trust this feed regardless”, so they can build their own packages with native binaries and publish their own peers for internal consumption.

I am not married to the naming of the peers, the format of the names, or anything like that. We can use it as a starting point of discussion, I just want the capability to exist.

[yishaigalatzer]

A couple more points to consider in the design

1. Runtime packages are currently (as of 3.5) only supported as a secondary package that is defined in the original packages, in other words packages can be sharded, but the original author has to do that. We only officially support the scenario in .net core libraries and not for 3rd party packages.
2. Runtime IDs have an inheritance behavior. For example win -> win7 -> win7-x86, the structure is defined in a runtime.json file that has to exist in a non RID specific package.
3. Exact match RIDs do not require pre-definition in a runtime.json. So for example a whole new platform "super-xbox-x128" would work out of the box without any need for a mapping.
4. Without server changes, restore will have to brute force search the whole inheritance graph. These graphs could be pretty deep (10-20 easily), so for existing servers this approach would be quite expensive. We will probably need to introduce a new server protocol/search capability to allow searching down the whole list - like id:Foo.Peer.* and let the client figure it out
5. This approach might also flood existing servers with a x10-x20 factor of calls until the new protocol is implemented. We should consider a mechanism to control the amount of searching this feature introduces. Perhaps a flag in the base package that tells restore that this package might bring runtime assets, and then "hope" that most packages will not look like that.

CC @ericstj @davidfowl @emgarten

[yishaigalatzer]

Another concern that comes to mind (from @emgarten) is the fact that restore will now have to find a way to not try and light up the better match package.

This is very similar to the 1.* floating package version scenario, except now we are going to have a floating mode by default as soon as a user installed a package like skia

[nguerrera]

Is the expectation that nuget restore automatically discovers the peer packages based on RID or would the user specify the peer packages that they want to pull in as inputs to restore?

In other words, would "SkiaSharp.Peer.PlayStation4” appear in the project or just something that says SkiaSharp and something that says Playstation4?

[mhutch]

Right now this is something you can do with RIDs, the problem is most platforms/frameworks don't currently support RIDs.