Add LoadKernelExtension, IsKernelExtensionAvailable

[fingolfin]

The idea is to provide "better" (?) APIs for packages to load kernel extensions. With IsKernelExtensionAvailable (and an upcoming kernel function to make it better), packages can simplify their TestAvailability functions, and at the same time make them more robust. Likewise, LoadKernelExtension simplifies init.g code.

The big win is that it'll then make it much easier for us to change how to locate and load kernel extensions; e.g. to better support make install of packages, or also better Julia integration.

Open questions:

1. perhaps I should rename these functions to LoadKernelModule and IsKernelModuleAvailable? Background: in the manual and various other places, we talk about "kernel modules". I (and some other places/people) use "kernel extension" (possibly a short hand for "kernel extension module") to distinguish it from built-in kernel modules; I guess one could also refer to it as "external kernel module" (but that would not be quite right for "static" kernel extension modules...).
2. In case of failure, perhaps IsKernelExtensionAvailable resp. IS_LOADABLE_DYN should return proper error codes (instead of just false) to indicate what exactly went wrong (file missing / file not a loadable module / file not a GAP kernel extension / kernel extension but built for wrong GAP version / ....). It could even be just a string; in any case, something that helps produce better error messages for AvailabilityTest in PackageInfo.g
3. Perhaps add an optional field KernelModule to the package info record, which when present would prompt GAP to automatically invoke IsKernelExtensionAvailable when checking the package availability; and LoadKernelExtension when loading the package (possible values: KernelModule := true = use package name, KernelModule := "modname"; or even KernelModule := ["mod1", "mod2"] if someone wants to support multiple kernel modules in one package. Alas, that seems very fringe, so I'd probably not do it)

[ThomasBreuer]

Nice idea. See my comments.

[ThomasBreuer]

Can the intended new entries in PackageInfo.g also replace the prerequisites.sh files used byBuildPackages.sh?

[fingolfin]

Hopefully / perhaps -- it depends a bit on what the package does in there...

[codecov]

Codecov Report

Merging #4231 (89e4961) into master (1dc1f81) will decrease coverage by 82.30%.
The diff coverage is n/a.

❗ Current head 89e4961 differs from pull request most recent head 5fff8c2. Consider uploading reports for the commit 5fff8c2 to get more accurate results

@@            Coverage Diff             @@
##           master   #4231       +/-   ##
==========================================
- Coverage   82.30%       0   -82.31%     
==========================================
  Files         678       0      -678     
  Lines      287790       0   -287790     
==========================================
- Hits       236876       0   -236876     
+ Misses      50914       0    -50914     

Impacted Files	Coverage Δ
src/modules.h
lib/cyclotom.gd
lib/helpdef.gi
lib/clas.gd
lib/cmdledit.g
lib/grpffmat.gd
lib/grppcfp.gi
lib/stbcbckt.gd
lib/ieee754.g
src/debug.c
... and 665 more

[fingolfin]

I've overhauled this now, and IMHO it is mostly ready, except for the lack of documentation.

Going beyond this, I wonder if we should add an optional field KernelExtension to the package info record. If that is present, then GAP could automatically invoke IsKernelExtensionAvailable when checking the package availability; and LoadKernelExtension when loading it. That said, perhaps this is too fringe?

[ChrisJefferson]

I generally like this PR. I have one (larger) comment -- I wonder if we should separate static and dynamic modules more?

Is there any evidence we can still (or demand for?) loading extensions as GAP static modules would work? The built-in ones (where we compile some library files with gac) are not handled in the "standard way", READ_GAP_ROOT calls LookupStaticModule. Therefore we could, perhaps, simplify this code some of the general loading code by removing mentions of static modules, and (even) rename 'static' to 'library' or something.

Of course, if people want the option in future of doing a big static build, where we static compile extensions, that wouldn't work.

[fingolfin]

@ChrisJefferson I am not sure what to make of your comment. It seems orthogonal to the content of this PR? If at all, this PR would help with "getting rid and/or renaming static modules": it removes the need to explicitly call into the APIs for static modules, by providing a fresh abstraction layer. So once we have this PR in, future packages could switch to using LoadKernelExtension and IsKernelExtensionAvailable. Then the package authors would not have to think about "static modules" anymore, even if they still wanted to support those.

The sooner we have this in, the sooner packages could start using it. Conversely, even if we get this into 4.12, it'll take time for this to be adopted by packages.

[fingolfin]

I notice that in the manual and various other places, we talk about "kernel modules". I (and some other places/people) use "kernel extension" (possibly a short hand for "kernel extension module") to distinguish it from built-in kernel modules; I guess one could also refer to it as "external kernel module" (but that would not be quite right for "static" kernel extension modules...).

Aaaanyway: perhaps I should rename these functions then to LoadKernelModule and IsKernelModuleAvailable ?

[fingolfin]

I also wonder if IsKernelExtensionAvailable resp. IS_LOADABLE_DYN should perhaps produce a proper error code (instead of just returning false) to indicate what exactly went wrong (file missing / file not a loadable module / file not a GAP kernel extension / kernel extension but built for wrong GAP version / ....). It could even be just a string; in any case, something that helps produce better error messages for AvailabilityTest in PackageInfo.g

[fingolfin]

Perhaps we should just merge this? There are plenty more things one could do, but this is already an improvement...