Disnix is free software; you can redistribute it and/or modify it under the terms of the GNU Lesser General Public License as published by the Free Software Foundation; either version 2.1 of the License, or (at your option) any later version. Disnix is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU Lesser General Public License for more details.

Deploying NixOS-based configurations is quite straight forward. A coordinator machine simply requires the presence of the Dysnomia and Disnix utilities in the system environment or a Nix user profile. Each target machine requires the presence of a Disnix service instance and (optionally) a number of container services.

environment.systemPackages = [ pkgs.dysnomia pkgs.disnix ];

$ nixos-rebuild switch

services.disnix.enable = true;

services.openssh.enable = true;

services.disnix.enableMultiUser = false;

services.mysql.enable = true;
services.tomcat.enable = true;

If it is desired to install Disnix manually, we must first install the Disnix and Dysnomia packages on every machine in the network. On each target machine, we must perform a number of additional steps to get the Disnix service to work properly.

$ ./configure options...
$ make
$ make install

$ ./configure --help

$ ./configure options...
$ make
$ make install

$ ./bootstrap

$ nix-env -i disnix

$ cp /usr/local/etc/dbus-1/system.d/disnix.conf /etc/dbus-1/system.d

<policy user="root">

<policy user="sander">

#!/bin/sh
# Start/stop the disnix-service.
#
### BEGIN INIT INFO
# Provides:          disnix-service
# Default-Start:     2 3 4 5
# Default-Stop:
# Short-Description: Disnix Service
# Description:       Exposes deployment operations to remote machines that are
#                    carried out by Nix and Dysnomia
### END INIT INFO

PATH=/home/sander/.nix-profile/bin:/bin:/usr/bin:/sbin:/usr/sbin
DESC="disnix service"
NAME=disnix-service
DAEMONUSER=sander
DAEMON=/home/$DAEMONUSER/.nix-profile/bin/disnix-service
PIDFILE=/var/run/disnix-service.pid
SCRIPTNAME=/etc/init.d/"$NAME"

test -f $DAEMON || exit 0

. /lib/lsb/init-functions

case "$1" in
start)  log_daemon_msg "Starting disnix service" "$NAME"
        start-stop-daemon --start --quiet --pidfile $PIDFILE --user $DAEMONUSER --name $NAME --background --chuid $DAEMONUSER --exec $DAEMON $EXTRA_OPTS
        log_end_msg $?
        ;;
stop)   log_daemon_msg "Stopping disnix service" "$NAME"
        start-stop-daemon --stop --quiet --user $DAEMONUSER --name $NAME --retry 5
        RETVAL=$?
        [ $RETVAL -eq 0 ] && [ -e "$PIDFILE" ] && rm -f $PIDFILE
        log_end_msg $RETVAL
        ;;
restart) log_daemon_msg "Restarting disnix service" "$NAME"
        $0 stop
        $0 start
        ;;
status)
        start-stop-daemon --status --pidfile $PIDFILE --name $NAME && exit 0 || exit $?
        ;;
*)      log_action_msg "Usage: /etc/init.d/disnix-service {start|stop|status|restart}"
        exit 2
        ;;
esac
exit 0

$ cygrunsrv -I dbus -p /usr/bin/dbus-daemon.exe -a '--system --nofork'

$ cygrunsrv -I disnix -p /usr/local/bin/disnix-service.exe \
  -e 'PATH=/bin:/usr/bin:/usr/local/bin' \
  -y dbus -u sander

$ editrights -u sander -l

$ editrights -u sander -a SeServiceLogonRight

$ mkdir -p /var/log/disnix
$ chown sander:users /var/log/disnix

3.2.7. Configuring the SSH protocol wrapper

Disnix also needs to be remotely connectible. In order to connect through SSH, you must install an SSH server, such as OpenSSH. Consult your system distribution's package manager for more information.

On Cygwin, we can configure SSH by running the following command-line instruction:

$ ssh-host-config

After configuring the services, you probably need to activate them for the fist time, which can be done by the Windows service manager (Control Panel -> System and Security -> Administrative Tools -> Services). You need to pick the Disnix service and select the start option. If you want to use the SSH server, you need to pick and start the 'CYGWIN sshd' service as well. A screenshot of this is shown in Figure 3.1, “Starting the Disnix Windows system service”.

Figure 3.1. Starting the Disnix Windows system service

mysqlUsername=root
mysqlPassword=secret
mysqlPort=3306
type=mysql-database

create table author
( AUTHOR_ID  INTEGER       NOT NULL,
  FirstName  VARCHAR(255)  NOT NULL,
  LastName   VARCHAR(255)  NOT NULL,
  PRIMARY KEY(AUTHOR_ID)
);

create table books
( ISBN       VARCHAR(255)  NOT NULL,
  Title      VARCHAR(255)  NOT NULL,
  AUTHOR_ID  VARCHAR(255)  NOT NULL,
  PRIMARY KEY(ISBN),
  FOREIGN KEY(AUTHOR_ID) references author(AUTHOR_ID) on update cascade on delete cascade
);

$ dysnomia --operation activate --component ./schema.sql --container mysql-database

In multi-user mode, only the super user and users who are members of the disnix group may access operations of the core Disnix service. In order to access the Disnix operations remotely, either an account with the right permissions is required or the protocol wrapper should perform the authentication to the core Disnix service.

The SSH wrapper, for instance, uses the credentials of the calling user on the coordinator by default. Therefore, every target machine requires the user to be defined in /etc/passwd and the user should be member of the disnix group.

In NixOS, the disnix user group is automatically added. For other systems this must be done by the system administrator. On most systems this user group can be added by typing:

$ groupadd disnix

A particular user can be made member of the disnix group by the following command-line instruction (someuser must be replaced by a desired username):

$ usermod -a -G disnix someuser

If an SSH connection is used, disnix-env may ask you to provide user credentials for each operation. This is not a bug, but an implication of using SSH. In order to make this process non-interactive, you must either generate an SSH keypair through ssh-keygen or use ssh-agent to remember the authentication settings.

When it is desired to use a single-user installation, the client must be instructed not to use the D-Bus service. This can be done by setting the following environment variable:

$ export DISNIX_REMOTE_CLIENT=disnix-run-activity

The example used in this section is a system which is used to manage staff of a fictional university. This system stores various kinds of records about staff members, such as their names, room numbers and IP addresses.

The system can determine a zipcode from the staff member's room number. From the zipcode it can determine the address of the building. From the IP address of a staff member, the system can determine the current location of the staff member. All the data repositories are stored in separate databases. Each data repository can be accessed through a web service.

Figure 4.1. Architecture of the StaffTracker

Figure 4.1, “Architecture of the StaffTracker” shows the architecture of the StaffTracker system. The architecture consists of three layers: a data layer, service layer and presentation layer. The database layer contains MySQL databases, storing data records such as zipcodes and staff members. The web service layer contains web service components exposing create, read, update and delete operations for each data set (the GeolocationService uses GeoIP to track a location). In the presentation layer, the StaffTracker web application front-end is shown, that can be used by end users to manage staff of a university.

All the components shown in the picture are distributable deployment units (or services). For instance, the GeolocationService can be deployed to a different machine in the network as the StaffTracker web application front-end. The arrows denote inter-dependency relationships. Inter-dependency correspond to network links between services. In this particular example, they are SOAP/HTTP or "plain" TCP connections.

The Nix package manager builds components from specifications called Nix expressions, written in the Nix expression language. Disnix also uses the Nix expression language to capture deployment aspects and defines service builds in a quite similar way. In this section, we first show how an ordinary Nix expression is written. Then we explain how this concept is extended to service-oriented systems.

{stdenv, fetchurl, perl}: 

stdenv.mkDerivation { 
  name = "hello-2.1.1"; 
  src = fetchurl { 
    url = ftp://ftp.gnu.org/gnu/hello/hello-2.1.1.tar.gz;
    md5 = "70c9ccf9fac07f762c24f2df2290784d";
  };
  buildInputs = [ perl ]; 

  meta = { 
    description = "GNU Hello, a classic computer science tool";
    homepage = http://www.gnu.org/software/hello/;
  };
}

rec { 
  stdenv = ...;
  
  fetchurl = ...;
  
  perl = import ../pkgs/perl { 
    inherit stdenv fetchurl;
  };
  
  hello = import ../pkgs/hello { 
    inherit stdenv fetchurl perl;
  };
  
  ...
}
				

rec {
  stdenv = ...;
  
  fetchurl = ...;
  
  perl = callPackage ../pkgs/perl { };
  
  hello = callPackage ../pkgs/hello { };
  
  ...
}
				

{stdenv, apacheAnt, axis2}: 
{zipcodes}: 

let
  contextXML = ''
    <Context>
      <Resource name="jdbc/ZipcodeDB" auth="Container" type="javax.sql.DataSource"
                maxActivate="100" maxIdle="30" maxWait="10000"
                username="${zipcodes.target.container.mysqlUsername}" password="${zipcodes.target.container.mysqlPassword}" driverClassName="com.mysql.jdbc.Driver"
                url="jdbc:mysql://${zipcodes.target.properties.hostname}:${toString (zipcodes.target.container.mysqlPort)}/${zipcodes.name}?autoReconnect=true" />
    </Context> 
  '';
in
stdenv.mkDerivation { 
  name = "ZipcodeService";
  src = ../../../../services/webservices/ZipcodeService;
  buildInputs = [ apacheAnt ];
  AXIS2_LIB = "${axis2}/lib";
  AXIS2_WEBAPP = "${axis2}/webapps/axis2";
  buildPhase = "ant generate.war";
  installPhase = ''
    ensureDir $out/conf/Catalina
    cat > $out/conf/Catalina/ZipcodeService.xml <<EOF 
    ${contextXML}
    EOF
    ensureDir $out/webapps
    cp *.war $out/webapps
  '';
}

{system, pkgs}:

rec { 
### Databases

  rooms = import ../pkgs/databases/rooms {
    inherit (pkgs) stdenv;
  };
  
  staff = import ../pkgs/databases/staff {
    inherit (pkgs) stdenv;
  };
  
  zipcodes = import ../pkgs/databases/zipcodes {
    inherit (pkgs) stdenv;
  };

### Web services + Clients
    
  ZipcodeService = import ../pkgs/webservices/ZipcodeService { 
    inherit (pkgs) stdenv apacheAnt axis2;
  };
  
  ZipcodeServiceClient = import ../pkgs/webservices/ZipcodeServiceClient {
    inherit (pkgs) stdenv apacheAnt axis2;
  };
  ...
  
### Web applications

  StaffTracker = import ../pkgs/webapplications/StaffTracker {
    inherit (pkgs) stdenv apacheAnt axis2;
    inherit GeolocationServiceClient RoomServiceClient StaffServiceClient ZipcodeServiceClient;
  };
}

{system, pkgs}:

let
  callPackage = pkgs.lib.callPackageWith (pkgs // self); 
  
  self = {
  ### Databases
    rooms = callPackage ../pkgs/databases/rooms { };
  
    staff = callPackage ../pkgs/databases/staff { };
  
    zipcodes = callPackage ../pkgs/databases/zipcodes { };

  ### Web services + Clients
    ZipcodeService = callPackage ../pkgs/webservices/ZipcodeService { };
  
    ZipcodeServiceClient = callPackage ../pkgs/webservices/ZipcodeServiceClient { };
    
    ...
  
  ### Web applications
    StaffTracker = callPackage ../pkgs/webapplications/StaffTracker { };
  };
}

{system, pkgs, distribution, invDistribution}: 

let customPkgs = import ../top-level/all-packages.nix { inherit system; }; 
in
rec { 
### Databases
  zipcodes = { 
    name = "zipcodes"; 
    pkg = customPkgs.zipcodes; 
    dependsOn = {};
    type = "mysql-database";
  };
  ...
  
### Web services  

  ZipcodeService = { 
    name = "ZipcodeService"; 
    pkg = customPkgs.ZipcodeService; 
    dependsOn = { 
      inherit zipcodes;
    };
    type = "tomcat-webapplication"; 
  };
  ...

### Web applications

  StaffTracker = {
    name = "StaffTracker";
    pkg = customPkgs.StaffTracker;
    dependsOn = {
      inherit GeolocationService RoomService StaffService ZipcodeService;
    };
    type = "tomcat-webapplication";
  };
}

{
  test1 = {
    properties = {
      hostname = "test1.example.org";
    };
    
    containers = {
      tomcat-webapplication = {
        tomcatPort = 8080;
      };
    };
    
    system = "i686-linux";
  };
  
  test2 = { 
    properties = { 
      hostname = "test2.example.org"; 
    };
    
    containers = { 
      tomcat-webapplication = {
        tomcatPort = 8080;
      };
      
      mysql-database = {
        mysqlPort = 3306; 
        mysqlUsername = "root";
        mysqlPassword = "admin";
      };
    };
    
    system = "x86_64-linux"; 
    numOfCores = 1; 
    targetProperty = "hostname"; 
    clientInterface = "disnix-ssh-client"; 
  }; 
}

{
  test1.properties.hostname = "test1.example.org";
  test2 = {
    properties.hostname = "test2.example.org";
    targetProperty = "hostname";
    clientInterface = "disnix-ssh-client";
  };
}

$ disnix-capture-infra infrastructure-basic.nix
{
  test1 = {
    properties = {
      "hostname" = "test1.example.org";
    };
    
    containers = {
      "tomcat-webapplication" = {
        "tomcatPort" = "8080";
      };
    };
  };
  
  test2 = {
    properties = {
      "hostname" = "test2.example.org";
    };
    
    containers = {
      "tomcat-webapplication" = {
        "tomcatPort" = "8080";
      };
      
      mysql-database = {
        "mysqlPort" = "3306";
        "mysqlUsername" = "root";
        "mysqlPassword" = "admin";
      };
    };
  };
}

{infrastructure}:

{
  zipcodes = [ infrastructure.test2 ]; 
  ZipcodeService = [ infrastructure.test1 ]; 
  StaffTracker = [ infrastructure.test1 infrastructure.test2 ]; 
  ...
}

We can perform a number of deployment activities with the deployment models shown earlier.

$ disnix-env -s services.nix -i infrastructure.nix -d distribution.nix

$ disnix-env --list-generations
   1   2018-02-13 13:22:35   (current)

ZipcodeService = [ infrastructure.test1 ];

ZipcodeService = [ infrastructure.test2 ];

$ disnix-env -s services.nix -i infrastructure.nix -d distribution.nix

   1   2018-02-13 13:22:35
   2   2018-02-13 13:30:24   (current)

$ disnix-env --rollback

$ disnix-env --switch-to-generation 1

$ disnix-env --build-on-targets -s services.nix -i infrastructure.nix -d distribution.nix

$ disnix-collect-garbage infrastructure.nix

$ disnix-collect-garbage -d infrastructure.nix

$ disnix-env --delete-generations old

$ nix-collect-garbage -d

$ disnix-env --delete-all-generations

{system, pkgs, distribution, invDistribution}:

let customPkgs = import ../top-level/all-packages.nix { inherit system; };
in
rec {
### Databases
  zipcodes = {
    name = "zipcodes";
    pkg = customPkgs.zipcodes;
    dependsOn = {};
    type = "mysql-database";
    deployState = true; 
  };
  ...
}

Example 5.1, “Annotated services model for the StaffTracker” shows a partial services model that is based on the services model of the StraffTracker example, described earlier in Example 4.7, “Services model for the StaffTracker”:

There is an important caveat when deploying multiple redundant instances of the same services. If state deployment has been enabled, Disnix assumes that redundant instances of the services all have the same state (that are synchronized somehow). If this is not the case, then you must give each service a unique identity.

Besides annotating individual services, it also possible to globally enable state deployment for all services without annotating them. See Chapter 10, Advanced options for more information.

With state management enabled, we can do various kinds of additional deployment tasks.

zipcodes = [ infrastructure.test2 ];

zipcodes = [ infrastructure.test1 ];

$ disnix-env -s services.nix -i infrastructure.nix -d distribution.nix

$ disnix-snapshot

$ disnix-restore

$ disnix-restore --no-upgrade

$ disnix-clean-snapshots --keep 3 infrastructure.nix

$ disnix-clean-snapshots --keep 0 infrastructure.nix

Although Disnix supports state deployment, it has been disabled by default. The reason why this is the default policy is that its mechanisms may neither be the preferred nor the optimal way of doing it.

Disnix uses Dysnomia's plugin system that invokes external tools taking care of snapshotting and restoring the state of the corresponding components. The good part of this approach is that it is general and works for all kinds of mutable components. Moreover, the tools that are consulted by the plugin system typically dump state in a portable and consistent format, which is also useful for a variety of reasons.

However, some of the drawbacks of this approach is that the process of snapshotting and restoring portable dumps is typically very slow, especially for large data sets. Moreover, the snapshot tools also write dumps entirely to the filesystem which is not always desired.

Alternative approaches of doing state deployment are the following:

This property actually stems from the way "ordinary packages" are built with the Nix package manager which is used as a basis for Disnix.

Nix package builds are influenced by its declared inputs only, such as the source code, build scripts and other kinds of dependencies, e.g. a compiler and libraries. Nix has means to ensure that undeclared dependencies cannot influence a build and that dependencies never collide with each other.

Moreover, since it does not matter where a package has been built, we can, for example, also download a package built from identical inputs from a remote location, instead of building it ourselves improving the efficiency of deployment processes.

In Disnix, Nix's concept of building packages has been extended to services in a distributed setting. The major difference between a package and a serivce is that services take an additional class of dependencies into account. Besides the intra-dependencies that Nix manages, services may also have inter-dependencies on services that may be deployed to remote machines in a network. Disnix can be used to configure services in such a way that a service knows how to reach them and that the system is activated and deactivated in the right order.

As a consequence, it does not take a machine's properties into account when deploying it to a target machine in the network unless a machine's properties are explicitly provided as dependencies of a service. In many cases, this is a considered a good thing. One of the things we could do is changing the location of the StaffTracker web application front-end service (the example shown in Chapter 4, A basic usage scenario), by changing the following line in the distribution model:

StaffTracker = [ infrastructure.test2 ];

to:

StaffTracker = [ infrastructure.test1 ];

Performing the redeployment procedure is actually quite efficient. Since the intra-dependencies and inter-dependencies of the StaffTracker service have not changed, we do not have to rebuild and reconfigure the StaffTracker service. We can simply take the existing build result from the coordinator machine (that has been previously distributed to machine test1) and distribute it to test2. Also, because the build result is the same, we have better guarantees that if the service worked on machine test1, it should work on machine test2 as well.

(As a sidenote: there is actually a situation in which a service will get rebuilt when moving it from one machine to another while its intra-dependencies and inter-dependencies have not changed. As shown in Example 4.8, “Infrastructure model” Disnix also supports heterogeneous service deployment meaning that we can run target machines having different CPU architectures and operating systems. For example, if test2 were a Linux machine and test1 a Mac OS X machine, Disnix attempts to rebuild it for the new platform. However, if all machines have the CPU architecture and operating system this will not happen).

Target-agnostic services are generally considered good because they improve reproducibility and efficiency when moving a service from machine to another. However, in some situations you may need to configure a service for a target machine specifically.

An example of a deployment scenario in which we need to deploy target-specific services, is when we want to deploy a collection of Node.js web applications and an nginx reverse proxy in which each web application should be reached by its own unique DNS domain name (e.g. http://webapp1.local, http://webapp2.local etc.).

This particular scenario is implemented as an example package, known as the Disnix virtual hosts example.

We could model the nginx reverse proxy and each web application as (target-agnostic) distributable services, and deploy them in a network with Disnix as follows:

Figure 6.1. Deployment architecture containing a single target-agnostic reverse proxy

We can declare the web applications to be inter-dependencies of the nginx service and generate its configuration accordingly.

Although this approach works, the downside is that in the above deployment architecture, the test1 machine has to handle all the network traffic including the requests that should be propagated to the web applications deployed to test2 making the system not very scalable, because only one machine is responsible for handling all the network load.

We can also deploy two redundant instances of the nginx service by specifying the following attribute in the distribution model:

nginx = [ infrastructure.test1 infrastructure.test2 ];

The above modification yields the following deployment architecture:

Figure 6.2. Deployment architecture containing multiple target-agnostic reverse proxies

The above deployment architecture is more scalable -- now requests meant for any of the web applications deployed to machine test1 can be handled by the nginx server deployed to test1 and the nginx server deployed to test2 can handle all the requests meant for the web applications deployed to test2.

Unfortunately, there is also an undesired side effect. As all the nginx services have the same form regardless to which machines they have been deployed, they have inter-dependencies on all web applications in the entire network including the ones that are not running on the same machine.

This property makes upgrading the system very inefficient. For example, if we update the webapp3 service (deployed to machine test2), the nginx configurations on all the other machines must be updated as well causing all nginx services on all machines to be upgraded, because they also have an inter-dependency on the upgraded web application.

In a 2 machine scenario with 4 web applications, this inefficiency may still be acceptable, but in a big environment with tens of web applications and tens of machines, we most likely suffer from many (hundreds of) unnecessary redeployment activities bringing the system down for a unnecessary long time.

A more efficient deployment architecture would be the following:

Figure 6.3. Deployment architecture containing multiple target-specific reverse proxies

We deploy two target-specific nginx services that only have inter-dependencies on the web applications deployed to the same machine. In this scenario, upgrading webapp3 does not affect the configurations of any of the services deployed to the test1 machine.

{pkgs, system, distribution, invDistribution}:

let
  customPkgs = ...
in
rec {
  ...

  nginx-wrapper-test1 = rec {
    name = "nginx-wrapper-test1";
    pkg = customPkgs.nginx-wrapper;
    dependsOn = {
      inherit webapp1 webapp2;
    };
    type = "wrapper";
  };

  nginx-wrapper-test2 = rec {
    name = "nginx-wrapper-test2";
    pkg = customPkgs.nginx-wrapper;
    dependsOn = {
      inherit webapp3 webapp4;
    };
    type = "wrapper";
  };
}

{infrastructure}:

{
  ...
  nginx-wrapper-test1 = [ infrastructure.test1 ];
  nginx-wrapper-test2 = [ infrastructure.test2 ];
}

{infrastructure}:

let
  inherit (builtins) listToAttrs attrNames getAttr;
in
{
  webapp1 = [ infrastructure.test1 ];
  webapp2 = [ infrastructure.test1 ];
  webapp3 = [ infrastructure.test2 ];
  webapp4 = [ infrastructure.test2 ];
} //

# To each target, distribute a reverse proxy

listToAttrs (map (targetName: {
  name = "nginx-wrapper-${targetName}";
  value = [ (getAttr targetName infrastructure) ];
}) (attrNames infrastructure))

{system, pkgs, distribution, invDistribution}:

let
  customPkgs = import ../top-level/all-packages.nix {
    inherit pkgs system;
  };
in
{
  webapp1 = ...
  
  webapp2 = ...
  
  webapp3 = ...
  
  webapp4 = ...
} //

# Generate nginx proxy per target host

builtins.listToAttrs (map (targetName:
  let
    serviceName = "nginx-wrapper-${targetName}";
    servicesToTarget = (builtins.getAttr targetName invDistribution).services;
  in
  { name = serviceName;
    value = {
      name = serviceName;
      pkg = customPkgs.nginx-wrapper;
      # The reverse proxy depends on all services distributed to the same
      # machine, except itself (of course)
      dependsOn = builtins.removeAttrs servicesToTarget [ serviceName ];
      type = "wrapper";
    };
  }
) (builtins.attrNames invDistribution))

{
  test1 = {
    services = {
      nginx-wrapper-test1 = {
        name = "nginx-wrapper-test1";
        pkg = customPkgs.nginx-wrapper;
        dependsOn = {
          inherit webapp1 webapp2;
        };
        type = "wrapper";
      };
      webapp1 = ...
      webapp2 = ...
    };
    properties = {
      hostname = "test1";
    }
  };
  
  test2 = {
    services = {
      nginx-wrapper-test2 = {
        name = "nginx-wrapper-test2";
        ...
      };
      webapp3 = ...
      webapp4 = ...
    };
    properties = {
      hostname = "test2";
    };
  };
}

Figure 7.1. Communication flow of the deployment operations

Figure 7.1, “Communication flow of the deployment operations” illustrates the communication flow of the disnix deployment utilities, such as disnix-env. When executing a deployment step that needs to be executed remotely, a connection client process is consulted that connects to a protocol wrapper on the remote machine. By default, Disnix uses disnix-ssh-client that connects through SSH. Other kinds of communication protocols are supported through extensions, such as SOAP/HTTP. See Chapter 10, Advanced options for more information.

The protocol wrapper connects to the core Disnix service, a D-Bus service exposing the deployment operations that Disnix needs to execute remotely. The core Disnix service invokes two external tools to carry out certain deployment activities. Nix is used for building and distributing packages. Dysnomia is used for activation, deactivation, locking, snapshotting, and restoring.

Figure 7.2. Architecture of the StaffTracker

As shown in Figure 7.2, “Architecture of the StaffTracker”, disnix-env, the command-line tool that performs all activities to make a service-oriented system available for use, is composed of several lower-level command-line tools each executing a specific deployment activity.

In the default workflow, the following command-line utilities are invoked:

Most of the deployment activities are encapsulated by a tool named: disnix-deploy that carries out a deployment process from a prebuilt Disnix manifest.

Two additional tools are used when the building on targets option has been enabled:

The build activity tools are encapsulated by a tool named: disnix-delegate that carries out the builds on the target machines from a provided services, infrastructure and distribution model.

When state deployment has been enabled, another three tools are invoked:

The state operations are encapsulated by a tool named: disnix-migrate that moves the state of services that have been moved from one machine to another.

As described earlier, we can also invoke the low-level utilities that disnix-env consults, to execute a certain deployment activity individually. In this section, we provide a few examples.

$ disnix-manifest -s services.nix -i infrastructure.nix -d distribution.nix

$ nix-store -qR ./result

$ disnix-instantiate -s services.nix -i infrastructure.nix -d distribution.nix

$ nix-store -qR ./result

$ disnix-build ./result

$ disnix-distribute ./result

$ disnix-activate ./result

In Nixpkgs, it is a common habit to write each package specification as a function in which the parameters denote intra-dependencies. In Chapter 4, A basic usage scenario we have shown that Disnix services follow the same convention and extend this approach with nested functions in which the inner function takes inter-dependencies into account.

For services that have no inter-dependencies, a Disnix expression is identical to an ordinary package expression. This means that, for example, an expression for a package such as the Midnight Commander shown in Example 8.1, “An example package expression and service expression with no inter-dependencies” is also a valid Disnix service with no inter-dependencies:

{ stdenv, fetchurl, pkgconfig, glib, gpm, file, e2fsprogs
, libX11, libICE, perl, zip, unzip, gettext, slang
}:

stdenv.mkDerivation {
  name = "mc-4.8.12";
  
  src = fetchurl {
    url = http://www.midnight-commander.org/downloads/mc-4.8.12.tar.bz2;
    sha256 = "15lkwcis0labshq9k8c2fqdwv8az2c87qpdqwp5p31s8gb1gqm0h";
  };
  
  buildInputs = [ pkgconfig perl glib gpm slang zip unzip file gettext
      libX11 libICE e2fsprogs ];

  meta = {
    description = "File Manager and User Shell for the GNU Project";
    homepage = http://www.midnight-commander.org;
    license = "GPLv2+";
    maintainers = [ stdenv.lib.maintainers.sander ];
  };
}  

Package and service expressions are functions that do not specify the versions or variants of the dependencies that should be used. To allow services to be deployed, we must compose them by providing the desired versions or variants of the dependencies as function parameters.

As shown in Chapter 4, A basic usage scenario we have to compose a Disnix service twice -- first its intra-dependencies in a composition expression and later its inter-dependencies in the services model.

{pkgs, system}:

let
  callPackage = pkgs.lib.callPackageWith (pkgs // self);

  self = {
    pkgconfig = callPackage ./pkgs/pkgconfig { };
  
    gpm = callPackage ./pkgs/gpm { };
  
    mc = callPackage ./pkgs/mc { };
  };
in
self
			

Example 8.2, “Composing packages locally” composes the Midnight Commander package by providing its intra-dependencies as function parameters. The third attribute (mc) invokes a function named: callPackage {} that imports the previous package expression and automatically provides the parameters having the same names as the function parameters. The callPackage { } function first consults the self attribute set (that composes some of Midnight Commander's dependencies as well, such as gpm and pkgconfig) and then any package from the Nixpkgs repository.

Previously, we have shown how to build packages from source code and its dependencies, and how to compose packages locally. For the deployment of services, more information is needed. For example, we need to compose their inter-dependencies so that services know how to reach them.

Furthermore, Disnix's end objective is to get a running service-oriented system and carries out extra deployment activities for services to accomplish this, such as activation and deactivation. The latter two steps are executed by a Dysnomia plugin that is determined by annotating a service with a type attribute.

For package deployment, specifying these extra attributes and executing these remaining activities are in principle not required. Nonetheless, we still need to provide a minimal services model so that Disnix knows which units can be deployed.

{pkgs, system, distribution, invDistribution}:

let
  customPkgs = import ./custom-packages.nix {
    inherit pkgs system;
  };
in
{
  mc = {
    name = "mc";
    pkg = customPkgs.mc;
    type = "package";
  };
}

In Example 8.3, “Exposing a package as a service” we import our intra-dependency composition expression and we use the pkg sub attribute to refer to the intra-dependency composition of the Midnight Commander. We annotate the Midnight Commander service with a package type to instruct Disnix that no additional deployment steps need to be performed beyond the installation of the package, such activation or deactivation.

Since the above pattern is common to all packages, we can also automatically generate services for any package in the composition expression, as shown in Example 8.4, “Exposing all locally composed packages as services”:

{pkgs, system, distribution, invDistribution}:

let
  customPkgs = import ./custom-packages.nix {
    inherit pkgs system;
  };
in
pkgs.lib.mapAttrs (name: pkg: {
  inherit name pkg;
  type = "package";
}) customPkgs

To allow users on the remote machines to conveniently access their packages, we must add Disnix' Nix profile to the PATH of a user on the remote machines:

$ export PATH=/nix/var/nix/profiles/disnix/default/bin:$PATH

When using NixOS, this variable can be extended by adding the following line to /etc/nixos/configuration.nix:

environment.variables.PATH = [ "/nix/var/nix/profiles/disnix/default/bin" ];

By providing an infrastructure model and distribution model, we can use Disnix to deploy packages to remote machines.

{
  test1.properties.hostname = "test1";
  test2 = {
    properties.hostname = "test2";
    system = "x86_64-darwin";
  };
}

Example 8.5, “A basic infrastructure model for package deployment” describes two machines that have hostname test1 and test2. Furthermore, machine test2 has a specific system architecture: x86_64-darwin that corresponds to a 64-bit Intel-based Mac OS X.

{infrastructure}:

{
  gpm = [ infrastructure.test1 ];
  pkgconfig = [ infrastructure.test2 ];
  mc = [ infrastructure.test1 infrastructure.test2 ];
}

In Example 8.6, “A basic distribution model for package deployment”, we distribute package gpm to machine test1, pkgconfig to machine test2 and mc to both machines.

When running the following command-line instruction:

$ disnix-env -s services.nix -i infrastructure.nix -d distribution.nix

Disnix executes all activities to get the packages in the distribution model deployed to the machines, such as building them from source code (including its dependencies), and distributing their dependency closures to the target machines. Because machine test2 may have a different system architecture as the coordinator machine, Disnix can use Nix's delegation mechanism to forward a build to a machine that is capable of doing it.

Alternatively, packages can also be built on the target machines through Disnix:

$ disnix-env --build-on-targets -s services.nix -i infrastructure.nix -d distribution.nix

After the deployment above command-line instructions have succeeded, we should be able to start the Midnight Commander on any of the target machines, by running:

$ mc

Besides deploying a custom set of packages, it is also possible to use Disnix to remotely deploy any package in the Nixpkgs repository, but doing so is a bit tricky. The main challenge lies in the fact that the Nix packages set is a nested set of attributes, whereas Disnix expects services to be addressed in one attribute set only.

Fortunately, the Nix expression language and Disnix models are flexible enough to implement a solution.

{infrastructure}:

{
  mc = [ infrastructure.test1 ];
  git = [ infrastructure.test1 ];
  wget = [ infrastructure.test1 ];
  "xlibs.libX11" = [ infrastructure.test1 ];
}

Example 8.7, “A distribution model referring to packages in Nixpkgs” shows a distribution model mapping a number of packages from the Nix packages repository to machines in the network. Note that we use a dot notation: xlibs.libX11 as an attribute name to refer to libX11 that can only be referenced as a sub attribute in Nixpkgs.

We can write a services model that uses the attribute names in the distribution model to refer to the corresponding package in Nixpkgs:

{pkgs, system, distribution, invDistribution}:

pkgs.lib.mapAttrs (name: targets:
  let
    attrPath = pkgs.lib.splitString "." name;
  in
  { inherit name;
    pkg = pkgs.lib.attrByPath attrPath
      (throw "package: ${name} cannot be referenced in the package set")
      pkgs;
    type = "package";
  }
) distribution

With Example 8.8, “A services model referring to packages in Nixpkgs” we can deploy any Nix package to any remote machine with Disnix.

Besides supporting single user installations, Nix also supports multi-user installations in which every user has its own private Nix profile with its own set of packages. With Disnix we can also manage multiple profiles. For example, by adding the --profile parameter, we can deploy another Nix profile that, for example, contains a set of packages for the user: sander:

$ disnix-env -s services.nix -i infrastructure.nix -d distribution.nix --profile sander

The user: sander can access its own set of packages by setting the PATH environment variable to:

$ export PATH=/nix/var/nix/profiles/disnix/sander:$PATH

Basically, every Dysnomia module implements a process, which takes two command-line parameters. The first parameter is one of the following:

The second parameter is a Nix path referring to a service. Moreover, all the container properties in the infrastructure model to which the service is deployed are passed as environment variables so that all relevant deployment properties are known.

#!/bin/bash
set -e
set -o pipefail
shopt -s nullglob

# Autoconf settings
export prefix=@prefix@

# Import utility functions
source @datadir@/@PACKAGE@/util

# Sets a number of common utility environment variables
composeUtilityVariables $0 $2 $3

case "$1" in
    activate) 
        # Initalize the given schema if the database does not exists
        if [ "$(echo "show databases" | @mysql@ --user=$mysqlUsername --password=$mysqlPassword -N | grep -x $componentName)" = "" ]
        then
            ( echo "create database $componentName;"
              echo "use $componentName;"
              
              if [ -d $2/mysql-databases ]
              then
                  cat $2/mysql-databases/*.sql
              fi
            ) | @mysql@ --user=$mysqlUsername --password=$mysqlPassword -N
        fi
        markComponentAsActive
        ;;
    deactivate) 
        markComponentAsGarbage
        ;;
    snapshot) 
        tmpdir=$(mktemp -d)
        cd $tmpdir
        @mysqldump@ --single-transaction --quick --user=$mysqlUsername --password=$mysqlPassword $componentName | head -n-1 | xz > dump.sql.xz
        
        hash=$(cat dump.sql.xz | sha256sum)
        hash=${hash:0:64}
        
        if [ -d $snapshotsPath/$hash ]
        then
            rm -Rf $tmpdir
        else
            mkdir -p $snapshotsPath/$hash
            mv dump.sql.xz $snapshotsPath/$hash
            rmdir $tmpdir
        fi
        createGenerationSymlink $snapshotsPath/$hash
        ;;
    restore) 
        lastSnapshot=$(determineLastSnapshot)
        
        if [ "$lastSnapshot" != "" ]
        then
            ( echo "use $componentName;"
              xzcat $snapshotsPath/$lastSnapshot/dump.sql.xz
            ) | @mysql@ --user=$mysqlUsername --password=$mysqlPassword -N
        fi
        ;;
    collect-garbage) 
        if componentMarkedAsGarbage
        then
            echo "drop database $componentName;" | @mysql@ --user=$mysqlUsername --password=$mysqlPassword -N
            unmarkComponentAsGarbage
        fi
        ;;
esac
			

Example 9.1, “MySQL database Dysnomia module” shows the Dysnomia module used for the mysql-database type:

Although the dysnomia package contains plugins for commonly found services, there may be special cases where the activation and deactivation procedure have to be executed directly by the services. For example, because there is no activation type available or the service requires a specialized activation procedure.

For these purposes, the wrapper type can be used. Basically, the wrapper module invokes the bin/wrapper executable in the Nix package with the first parameter passed to the activation script (which is activate, deactivate etc.). Then this process performs all the steps to activate the component and so on.

#!/bin/bash -e

export PATH=@PREFIX@/bin:$PATH

case "$1" in
    activate)
        nohup disnix-tcp-proxy @srcPort@ @destHostname@ @destPort@ /tmp/disnix-tcp-proxy-@srcPort@.lock > /var/log/$(basename @PREFIX@).log & pid=$!
        echo $pid > /var/run/$(basename @PREFIX@).pid
        ;;
    deactivate)
        kill $(cat /var/run/@PREFIX@.pid)
        rm -f /var/run/$(basename @PREFIX@).pid
        ;;
    lock)
        if [ -f /tmp/disnix-tcp-proxy-@srcPort@.lock ]
        then
            exit 1
        else
            touch /tmp/disnix-tcp-proxy-@srcPort@.lock
            
            if [ -f /var/run/$(basename @PREFIX@).pid ]
            then
                while [ "$(disnix-tcp-proxy-client)" != "0" ]
                do
                    sleep 1
                done
            fi
        fi
        ;;
    unlock)
        rm -f /tmp/disnix-tcp-proxy-@srcPort@.lock
        ;;
esac
			

Example 9.2, “Disnix TCP proxy wrapper script” shows an example of a wrapper script used for the TCP proxy example available from my Github page. As you may notice, the structure is very similar to an activation script, since it also contains the activate and deactivate steps.

Moreover, this wrapper script also implements a lock and unlock step, which notify the service that an upgrade is starting (a phase in which services are deactivated and activated and temporarily makes certain parts of the system unavailable for use) and that the upgrade phase is finished.

As mentioned before, the Disnix service consists of a core service and a protocol wrapper. By default, an SSH wrapper is used, but other types of wrappers can be used as well, such as SOAP, provided by the external DisnixWebService package.

The coordinator machine invokes an external process which performs communication with the Disnix service. By default disnix-ssh-client is consulted. A different client can be used by either setting the DISNIX_CLIENT_INTERFACE environment variable with the path to the executable or by using the --interface command-line option, for commands such as disnix-env.

For example, by specifying:

$ export DISNIX_CLIENT_INTERFACE=disnix-soap-client

The disnix-soap-client command is used to communicate with a remote Disnix service.

Apart from configuring the coordinator machine, each target machine must also run the connection wrapper so that it can use the given protocol. Refer to the documentation of the extension for specific instructions.

Another wrapper is disnix-client that connects directly to the D-Bus system bus to invoke Disnix service operations. This wrapper is useful for debugging purposes, but you cannot use this client for remote connections.

In some cases, also the target property must be configured. By default, Disnix uses the hostname property in the infrastructure model, to determine how to connect to the remote Disnix service in order to perform remote deployment steps. This property is not suitable for every protocol. A web service interface interace, for example, requires an URL.

The connection attribute can be changed by either setting the DISNIX_TARGET_PROPERTY environment variable with the attribute name that contains the address of the remote Disnix service or by using the --target-property command-line option.

For example, by specifying:

$ export DISNIX_TARGET_PROPERTY=sshTarget

The sshTarget attribute defined in the infrastructure model is used to determine the address of the Disnix service.

It is also possible to define a target property and client interface for each individual machine to support multi connection protocol deployment. See Example 4.8, “Infrastructure model” for more information.

By default, Disnix assumes that the models that you are currently using represent one particular distributed environment. You can also use multiple profiles, which allow you to maintain multiple distributed system environments from one coordinator machine. By using the --profile option for the disnix-env and disnix-activate commands, you can specify which profile you want to use, so that they do not interfere with each other.

The following instructions will install a particular distributed environment:

$ disnix-env -s my-default-services.nix -i my-default-infrastructure.nix -d my-default-distribution.nix

By running the following command with three models of another distributed environment:

$ disnix-env -s my-other-services.nix -i my-other-infrastructure.nix -d my-other-distribution.nix

Disnix will upgrade the previous the default environment to match the models defined in the other environment, which is not desirable. However, by using the --profile option Disnix deploys the new distributed system without looking to the default system's deployment state and maintains two seperate configuration next to each other:

$ disnix-env --profile other -s my-other-services.nix -i my-other-infrastructure.nix -d my-other-distribution.nix

Besides using the --profile option, you can also use an environment variable:

$ export DISNIX_PROFILE=other

If it desired to let Disnix manage state, you must annotate the corresponding services in the service model. However, it is also possible to override Disnix's default behaviour to enable state manage management for all services by default.

Global state deployment can be enabled by providing the --deploy-state command-line option to commands such as disnix-env or by setting the following environment variable:

$ export DISNIX_DEPLOY_STATE=1

As described in Chapter 4, A basic usage scenario, when mapping a service to a target machine in the distribution model, Disnix automatically maps the service to the appropriate container, by referring to a container with the same name as the type the service belongs to.

In some unconventional scenarios, it may be desired to run multiple instances of the same container on one machine. In such cases, automapping no longer works and a different notation is required. In Disnix, the following mapping in the distribution model (mapping a service to a list of target machines in the infrastructure model):

ZipcodeService = [ infrastructure.test2 ];

Is equivalent to the mapping in the following notation:

ZipcodeService = {
  targets = [ { target = infrastructure.test2; } ];
};

In the above notation, we define an attribute set in which the targets property refers to a list of attribute sets defining all possible attributes of a mapping. This alternative notation is more verbose and allows more mapping properties to be specified.

{
  test1 = {
    properties = {
      hostname = "test1.example.org";
    };
    
    containers = {
      tomcat-first = {
        tomcatPort = 8080;
      };
      
      tomcat-second = {
        tomcatPort = 8081;
      };
    };
  };
  
  test2 = {
    properties = {
      hostname = "test2.example.org";
    };
    
    containers = {
      mysql-first = {
        mysqlPort = 3306;
        mysqlUsername = "root";
        mysqlPassword = "admin";
      };
      
      mysql-second = {
        mysqlPort = 3307;
        mysqlUsername = "root";
        mysqlPassword = "secret";
      };
    };
  }; 
}

Example 10.1, “Infrastructure model with multiple container instances” is based on the StaffTracker infrastructure model shown in Example 4.8, “Infrastructure model”. In this modified infrastructure model, the test1 machine hosts two Apache Tomcat servers (one listening on TCP port 8080 and the other on TCP port 8081) and the test2 machine hosts two MySQL DBMSes (one listening on TCP port 3306 and the other on TCP port 3307). Because we have two instances of each container and their names do not correspond to the services types, automapping no longer works.

{infrastructure}:

{
  zipcodes = {
    targets = [ { target = infrastructure.test2; container = "mysql-first"; } ]; 
  };
  ZipcodeService = {
    targets = [ { target = infrastructure.test1; container = "tomcat-second"; } ]; 
  };
  StaffTracker = {
    targets = [ 
      { target = infrastructure.test1; container = "tomcat-first"; }
      { target = infrastructure.test1; container = "tomcat-second"; }
    ];
  };
  ...
}

Example 10.2, “Distribution model for the StaffTracker mapping to multiple containers” shows a distribution model using the alternative notation to directly control the container mappings:

When running production systems, you cannot get around unforeseen incidents and problems, such as crashes and database inconsistencies. To make diagnosing errors and executing arbitrary maintenance tasks more convenient, the disnix-diagnose can be used. The purpose of this tool is to spawn remote shell sessions containing all relevant configuration settings as environment variables. Furthermore, it will display command-line suggestions with command maintenance tasks.

To spawn a diagnostic interactive shell for a service, such as the staff MySQL database, simply run:

$ disnix-diagnose -S staff

The above command will query the deployment configuration of the system, remotely connects to the machine (typically through SSH) and spawns a diagnostic shell session.

It is also possible to execute arbitrary shell commands in the session. For example, the following command will query all staff records:

$ disnix-diagnose -S staff --command 'echo "select * from staff" | mysql -u $mysqlUsername -p $mysqlPassword staff'

Disnix can also host redundant instances of the same service. In such cases, you must refine the search query with a target or container parameter. The following instruction specifies that we want to connect to the StaffTracker service deployed to machine test1:

$ disnix-diagnose -S StaffTracker --target test1

It is still possible to execute remote shell commands for redundantly deployed instances. For example, the following command may be executed several times:

$ disnix-diagnose -S StaffTracker --command 'echo I may see this message multiple times!'

In some cases, you may want to execute other kinds of maintenance tasks or you simply want to know where a particular service resides. This can be done by running the following command:

$ disnix-diagnose -S StaffTracker --show-mappings

As a remark: the diagnose tool does not work with all Disnix client instances. Currently, only connectors that use SSH are supported!

As mentioned before, inter-dependencies serve two purposes -- to allow services to find them (e.g. by propagating their connection attributes) and to ensure that services are activated in the right order. It is not allowed that a service gets activated before any of its inter-dependencies.

Activation order strictness is generally a good property, but it comes at a price. It makes certain kinds of upgrades very expensive. For example, when a service has to be updated, then all interdependent services need to be updated as well. For a services that has many interdependent services, this could become quite expensive.

When a connection is not considered to be critical, e.g. if a disconnection for a short time does not bring the service's functionality down in a harmful way, it is possible to disregard the activation order, so that upgrades become less expensive and faster.

StaffTracker = {
  name = "StaffTracker";
  pkg = customPkgs.StaffTracker;
  dependsOn = {
    inherit RoomService StaffService ZipcodeService;
  };
  connectsTo = {
    inherit GeolocationService;
  };
  type = "tomcat-webapplication";
};

To disregard the activation order, you can annotate a service with the connectsTo property instead of the dependsOn property. In Example 10.3, “A service disregarding the activation order”, the connection to the GeolocationService is considered to be non-critial and Disnix will drop the guarantee that it will be deployed before the StaffTracker. Furthermore, the StaffTracker service will not be reactivated if the GeolocationService gets updated.

Disregarding the order has another useful property. When two services mutually depend on each other they can also be declared inter-dependencies. This is particularly useful for systems that have a ring structure.

In some cases, other communication kinds of communication protocols are needed to connect to remote machines besides SSH. The DisnixWebService is a package that implements a SOAP interface for deployment operations and a client interface named disnix-soap-client to connect to it.

Although Disnix manages the distributable components (services) of which a distributed system is composed and the inter-dependencies, Disnix does not manage the underlying system configurations of the actual machines to which services are deployed. The DisnixOS package combines the Disnix tooling for service deployment and NixOS tooling to deploy the underlying infrastructure.

Furthermore, it integrates with the NixOS test driver that can be used to automatically create virtual machine instances from the network configuration for testing.

Disnix implements a static deployment model -- to make it work it requires a infrastructure model and distribution model to be manually specified every time something in the configuration changes.

Dynamic Disnix is an extended toolset providing a discovery service to capture machine configurations in a network, as well as a distribution generation framework that can be used to generate a distribution model from technical and non-functional properties of services and machines.

These tools can be used to develop a framework enabling self adaptive (re)deployment of service-oriented systems.