Proposal: Simplify Nixago by introducing engines

[jmgilman]

The current structure of Nixago presents the following problems:

1. Generation of configuration files is tightly bound to CUE. This often comes as a surprise as, at face value, it's not apparent what Nixago is using for configuration generation under the hood. Additionally, it requires learning a new language that does not currently have wide adoption (although it is growing).

2. Plugin development is tightly bound to the Nixago repository. The external interface for Nixago (make) explicitly asks for a plugin name to use (although providing an optional plugin parameter is in the roadmap with Allow users to specify their own plugin #22). This provides a barrier to rapid adoption as an end-user is not likely to want to immediately contribute to plugin development just to start using Nixago.

3. The relationship between the end-user, plugins, and the CUE backend is convoluted. While a diagram does help to understand it, it's ripe for refactoring to simplify the internal interfaces. Nixago currently has an identity crisis regarding what exactly is it trying to accomplish.

The proposal tracked by this issue contains a solution that addresses the above three problems.

Proposal

As noted in the problem section, Nixago currently faces an identity crisis. It needs to be streamlined to "do one thing and do it well." To assist in resolving this problem, the following sections help to clarify the direction going forward.

What does Nixago provide?

Nixago is a framework for generating and managing configuration files. It addresses any situation which has the following two requirements:

1. Templating a configuration file using a Nix expression as an input
2. Managing the generated file on the local filesystem

The primary problem being targeted is the abundance of configuration files that build-up at the root of a repository.

How does Nixago accomplish this?

Nixago provides a single external interface that ingests user configuration data and produces a combination of a derivation that builds the configuration file and a shell hook that manages it. The interface uses a well-defined module for collecting information from the user and then executes upon it to produce the desired result. The contractual nature of the module interface allows third parties to build additional tooling which can further simplify the external interface (i.e., ingests minimal information and produces a compatible output for Nixago).

Components

Nixago consists of three primary components: the external interface (function), the request module, and the engines.

External Interface

Nixago exposes a single external interface to end-users in the form of a function (make). This is the only entry-point into Nixago. The make function accepts a single attribute set that adheres to the contract specified by the request module. The user specifies one or more options provided by the module and Nixago executes upon it, returning the desired result.

The predictability and standardization afforded by this external interface allow further growth through third-party integrations. A developer may create additional tooling which produces an output that adheres to the contract and thus further simplifies the process of generating a configuration file. This is largely inspired by the Unix philosophy and is analogous to piping another tool between the end-user and Nixago.

Request Module

The request module provides a concrete model with which internal parts of Nixago can collectively manage a request from the user. It contains all required information to reproducibly generate a configuration file and shell hook.

The interface for the module is as follows:

Parameter	Required	Description
configData	Yes	The raw configuration data
output	Yes	The relative path to link the generated file
format	Yes	The format of the output file
root	No	The root path from which the relative path is derived (defaults to PRJ_ROOT)
engine	No	The engine to use for generation (defaults to engines.nix)

Engine

An engine is the largest internal component of Nixago and is responsible for producing a derivation that generates the configuration file. This is the only requirement of an engine. An engine has two sub-components: a submodule that declares its interface and a function that will be invoked by Nixago.

Each engine will define a submodule that will integrate into the request module and serve as the point of entry for configuring the engine. The submodule definition is not standardized. Attempts to standardize such a sporadic interface are likely to result in a disorganized interface full of hacks to provide support for each included engine. Thus, an end-user must be aware of which engine they want to use and what configuration options are available for it.

Each engine will additionally provide a single function that returns a derivation. The function will accept a single input: an attribute set that satisfies the request module interface. In return, the engine must produce a derivation that builds the desired configuration file.

Shell Hook

In addition to the above components, Nixago will generate a shell hook that manages the derivation returned by an engine. The hook will be responsible for either linking the result locally or maintaining a local copy of it.

Example

{
  configData = {
    myvar1 = "value1";
    myvar2 = true;
  };
  format = "ini";
  output = "config.ini";
}

This represents the most basic example achievable with Nixago. It will return a derivation that generates an INI file at $PRJ_ROOT/config.ini. The default engine used by Nixago is nix which utilizes the pkgs.formats functions to generate output data. No additional options are required to configure this engine and therefore the end-user may opt to exclude it.

{
  configData = {
    myvar1 = "value1";
    myvar2 = true;
  };
  output = "config.ini";
  engine = nixago.engines.cue {
    files = [ ./template.cue ];
    package = "mypackage";
  };
}

This example uses the CUE engine to accomplish the same result as above. The CUE engine has its own interface which requires a list of CUE files to parse and the optional package to use.

Diagram

%%{ init : { "flowchart" : { "curve" : "linear" }}}%%
%% 
flowchart TD
    flake{{User's flake.nix}}
    make(Call to make)
    engine(Call to engine)
    flake -- request ---> make
    make <-- request ---> engine
    make -- result --> flake

Loading

[nixos-discourse]

This issue has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/validate-and-generate-configuration-files-using-nix-and-cue/19326/6