Community modules

[tzarc]

Description

Add support for community modules. Community modules are intended to be an area which provides common functionality to be easily imported into a keymap.

Additionally, community modules support placement inside userspace; module developers can thus create modules in separate repositories, and users can subsequently add them as userspace-local submodules to make them available for use in user keymaps.

• keyboard.json, info.json, and keymap.json now accept a top-level array of strings called "modules"
• Keyboard-imported modules precede keymap-imported modules.
• Module implementations are able to hook an initial set of key APIs within QMK, now allowing for the sequence _modules => _kb => _user.
• Each module has an equivalent hook API generated and invoked inside _modules, in the module order provided.
  • For a single API example, the hello_world module has the following overridable functions defined:
    • process_record_hello_world_user -- a no-op but allows user keymaps to hook the module
    • process_record_hello_world_kb -- invokes _user by default, follows the same callback ordering as the rest of QMK
    • process_record_hello_world -- the actual module implementation of process_record_xxxx, executed before process_record_kb or process_record_user.
• The full list of hookable APIs will be present in the documentation when written.

Types of Changes

• Core
• Bugfix
• New feature
• Enhancement/optimization
• Keyboard (addition or update)
• Keymap/layout (addition or update)
• Documentation

Checklist

• My code follows the code style of this project: C, Python
• I have read the PR Checklist document and have made the appropriate changes.
• My change requires a change to the documentation.
• I have updated the documentation accordingly.
• I have read the CONTRIBUTING document.
• I have added tests to cover my changes.
• I have tested the changes and verified that they work and don't break anything (as well as I can manage).

[getreuer]

Very cool! I'm excited to see ASSERT_COMMUNITY_MODULES_MIN_API_VERSION. With my userspace libs, I do a hacky preprocessor check to detect very old QMK versions:

#if !defined(IS_QK_MOD_TAP)
// Attempt to detect out-of-date QMK installation, which would fail with
// implicit-function-declaration errors in the code below.
#error "achordion: QMK version is too old to build. Please update QMK."
#else
// Achordion implementation...

ASSERT_COMMUNITY_MODULES_MIN_API_VERSION is way better =)

[getreuer]

For a keymap using multiple modules at once, how do you call the process_record, etc. hooks for every module? I was expecting a "call next module" or a loop somewhere but didn't find how you do it.

[tzarc]

Modules calls kb calls user. To facilitate users/kb overriding the module, each module has its own module_user and module_kb, called in the usual way. The codegen handles the base implementation core-side, so user/kb-level code doesn’t need to call modules as it’s the new “parent” — each module call is chained within the codegen.

[tzarc]

To elaborate, at the moment codegen generates the following invocation styles:

void keyboard_post_init_modules(void) {
    keyboard_post_init_hello_world2();
    keyboard_post_init_hello_world();
}

bool pre_process_record_modules(uint16_t keycode, keyrecord_t *record) {
    return true
        && pre_process_record_hello_world2(keycode, record)
        && pre_process_record_hello_world(keycode, record)
    ;
}

As well as the module-specific callbacks:

__attribute__((weak)) bool pre_process_record_hello_world_user(uint16_t keycode, keyrecord_t *record) {
    return true;
}

__attribute__((weak)) bool pre_process_record_hello_world_kb(uint16_t keycode, keyrecord_t *record) {
    if(!pre_process_record_hello_world_user(keycode, record)) { return false; }
    return true;
}

__attribute__((weak)) bool pre_process_record_hello_world(uint16_t keycode, keyrecord_t *record) {
    if(!pre_process_record_hello_world_kb(keycode, record)) { return false; }
    return true;
}

The user/kb can override the first two, with the module implementing the third as necessary. If unused, then the optimiser just skips the lot.

[getreuer]

I see range QK_COMMUNITY_MODULE to QK_COMMUNITY_MODULE_MAX is reserved for modules, which is helpful. If multiple modules are used at once, and each needs a few keycodes, is there a way that each module could get part of this range without overlapping with other modules? It would be great if modules could each allocate keycodes in a standardized way, essentially, an automatic way to enumerate the range like:

enum module_keycodes {
  // Keycodes for module Foo.
  FOO_KC1 = QK_COMMUNITY_MODULE,
  FOO_KC2,
  FOO_KC3,
  FOO_KC4,
  // Keycodes for module Bar.
  BAR_KC1,
  BAR_KC2,
  // Keycodes for more modules...
};

[tzarc]

All module keycodes are assigned automatically, with keyboard module keycodes preceding keymap module keycodes, in the module order specified in the keyboard or keymap json.

Basically does exactly what you specified but with stable ordering. The define is mainly just to allocate a range, rather than using the keycode directly.