Merge pull request #600 from PX4/posix_shell_docs

add documentation for the Posix shell

en/SUMMARY.md

@@ -27,6 +27,7 @@
   * [Flight Modes](concept/flight_modes.md)
   * [Mixing and Actuators](concept/mixing.md)
   * [PWM limit state machine](concept/pwm_limit.md)
+  * [System Startup](concept/system_startup.md)
 * [Simulation](simulation/README.md)
   * [jMAVSim Simulation](simulation/jmavsim.md)
   * [Gazebo Simulation](simulation/gazebo.md)
@@ -100,7 +101,6 @@
   * [Motion Capture \(VICON, Optitrack\)](tutorials/motion-capture-vicon-optitrack.md)
   * [S.Bus Driver for Linux](tutorials/linux_sbus.md)
 * [Advanced Topics](advanced/README.md)
-  * [System Boot](advanced/system_startup.md)
   * [Parameters & Configs](advanced/parameters_and_configurations.md)
   * [Parameter Reference](advanced/parameter_reference.md)
   * [Installing driver for Intel RealSense R200](advanced/realsense_intel_driver.md)

en/advanced/gimbal_control.md

@@ -41,7 +41,7 @@ The output assignment is as following:
 > of how mixers work and the format of the mixer file.
 
 The outputs can be customized by [creating a mixer
-file](../advanced/system_startup.md#starting-a-custom-mixer) on the SD card with
+file](../concept/system_startup.md#starting-a-custom-mixer) on the SD card with
 name `etc/mixers/mount.aux.mix`.
 
 A basic basic mixer configuration for a mount is shown below.

en/advanced/parameters_and_configurations.md

@@ -2,7 +2,7 @@
 
 The PX4 platform uses the param subsystem (a flat table of float and int32_t values) and text files (for mixers and startup scripts) to store its configuration.
 
-The [system startup](../advanced/system_startup.md) and how [airframe configurations](../airframes/adding_a_new_frame.md) work are detailed on other pages. This section discusses the param subsystem in detail
+The [system startup](../concept/system_startup.md) and how [airframe configurations](../airframes/adding_a_new_frame.md) work are detailed on other pages. This section discusses the param subsystem in detail
 
 ## Command Line usage
 

en/airframes/adding_a_new_frame.md

@@ -4,7 +4,7 @@ PX4 uses canned airframe configurations as starting point for airframes. The con
 
 Adding a configuration is straightforward: create a new config file in the [init.d folder](https://github.com/PX4/Firmware/tree/master/ROMFS/px4fmu_common/init.d) (prepend the filename with an unused autostart ID), then [build and upload](../setup/building_px4.md) the software.
 
-Developers who do not want to create their own configuration can instead customize existing configurations using text files on the microSD card, as detailed on the [custom system startup](../advanced/system_startup.md) page.
+Developers who do not want to create their own configuration can instead customize existing configurations using text files on the microSD card, as detailed on the [custom system startup](../concept/system_startup.md) page.
 
 ## Configuration File Overview
 

en/apps/module_template.md

@@ -11,7 +11,7 @@ The template demonstrates the following additional features/aspects that are req
 - uORB subscriptions and waiting for topic updates.
 - Controlling the task that runs in the background via `start`/`stop`/`status`.
   The `module start [<arguments>]` command can then be directly added to the
-  [startup script](../advanced/system_startup.md).
+  [startup script](../concept/system_startup.md).
 - Command-line argument parsing.
 - Documentation: the `PRINT_MODULE_*` methods serve two purposes (the API is
   documented [in the source code](https://github.com/PX4/Firmware/blob/v1.8.0/src/platforms/px4_module.h#L381)):

en/concept/system_startup.md

@@ -0,0 +1,108 @@
+# System Startup
+
+The PX4 startup is controlled by shell scripts.
+On NuttX they reside in the [ROMFS/px4fmu_common/init.d](https://github.com/PX4/Firmware/tree/master/ROMFS/px4fmu_common/init.d) folder - some of these are also used on Posix (Linux/MacOS). The scripts that are only used on Posix are located in [ROMFS/px4fmu_common/init.d-posix](https://github.com/PX4/Firmware/tree/master/ROMFS/px4fmu_common/init.d-posix).
+
+All files starting with a number and underscore (e.g. `10000_airplane`) are canned airframe configurations. They are exported at build-time into an `airframes.xml` file which is parsed by [QGroundControl](http://qgroundcontrol.com) for the airframe selection UI. Adding a new configuration is covered [here](../airframes/adding_a_new_frame.md).
+
+The remaining files are part of the general startup logic. The first executed file is the [init.d/rcS](https://github.com/PX4/Firmware/blob/master/ROMFS/px4fmu_common/init.d/rcS) script (or [init.d-posix/rcS](https://github.com/PX4/Firmware/blob/master/ROMFS/px4fmu_common/init.d-posix/rcS) on Posix), which calls all other scripts.
+
+The following sections are split according to the operating system that PX4 runs on.
+
+
+## Posix (Linux/MacOS)
+
+On Posix, the system shell is used as script interpreter (e.g. bash).
+For that to work, a few things are required:
+- PX4 modules need to look like individual executables to the system. This is done via symbolic links.
+  For each module a symbolic link `px4-<module> -> px4` is created in the `bin` directory of the build folder.
+  When executed, the binary path is checked (`argv[0]`), and if it is a module (starts with `px4-`), it sends the command to the main px4 instance (see below).
+  > **Tip** The `px4-` prefix is used to avoid conflicts with system commands (e.g. `shutdown`), and it also allows for simple tab completion by typing `px4-<TAB>`.
+- The shell needs to know where to find the symbolic links. For that the `bin` directory with the symbolic links is added to the `PATH` variable right before executing the startup scripts.
+- The shell starts each module as a new (client) process. Each client process needs to communicate with the main instance of px4 (the server), where the actual modules are running as threads.
+  This is done with [FIFOs (also called named pipes)](http://man7.org/linux/man-pages/man7/fifo.7.html).
+  The server has a FIFO opened, on which clients can send commands.
+  In addition each client opens its own FIFO, which the server then uses to send information to the client (strings printed to the console for example).
+- The startup scripts call the module directly, e.g. `commander start`, rather than using the `px4-` prefix. This works via aliases: for each module an alias in the form of `alias <module>=px4-<module>` is created in the file `bin/px4-alias.sh`.
+- The `rcS` script is executed from the main px4 instance.
+  It does not start any modules, but first updates the `PATH` variable and then simply runs a shell with the `rcS` file as argument.
+- In addition to that, multiple server instances can be started for multi-vehicle simulations. A client selects the instance via `--instance`. The instance is available in the script via `$px4_instance` variable.
+
+The modules can be executed from any terminal when PX4 is already running on a system. For example:
+```
+cd <Firmware>/build/posix_sitl_default/bin
+./px4-commander takeoff
+./px4-listener sensor_accel
+```
+
+## NuttX
+NuttX has an integrated shell interpreter ([NSH](http://nuttx.org/Documentation/NuttShell.html)), and thus scripts can be executed directly.
+
+### Debugging the System Boot
+
+A failure of a driver of software component will not lead to an aborted boot. This is controlled via `set +e` in the startup script.
+
+The boot sequence can be debugged by connecting the [system console](../debug/system_console.md) and power-cycling the board. The resulting boot log has detailed information about the boot sequence and should contain hints why the boot aborted.
+
+#### Common boot failure causes
+
+  * For custom applications: The system was out of RAM. Run the `free` command to see the amount of free RAM.
+  * A software fault or assertion resulting in a stack trace
+
+### Replacing the System Startup
+
+In most cases customizing the default boot is the better approach, which is documented below. If the complete boot should be replaced, create a file `/fs/microsd/etc/rc.txt`, which is located in the `etc` folder on the microSD card. If this file is present nothing in the system will be auto-started.
+
+### Customizing the System Startup
+
+The best way to customize the system startup is to introduce a [new airframe configuration](../airframes/adding_a_new_frame.md). If only tweaks are wanted (like starting one more application or just using a different mixer) special hooks in the startup can be used.
+
+> **Caution** The system boot files are UNIX FILES which require UNIX LINE ENDINGS. If editing on Windows use a suitable editor.
+
+There are three main hooks. Note that the root folder of the microsd card is identified by the path `/fs/microsd`.
+
+  * /fs/microsd/etc/config.txt
+  * /fs/microsd/etc/extras.txt
+  * /fs/microsd/etc/mixers/NAME_OF_MIXER
+
+#### Customizing the Configuration (config.txt)
+
+The `config.txt` file can be used to modify shell variables. It is loaded after the main system has been configured and *before* it is booted.
+
+#### Starting additional applications
+
+The `extras.txt` can be used to start additional applications after the main system boot. Typically these would be payload controllers or similar optional custom components.
+
+> **Caution** Calling an unknown command in system boot files may result in boot failure. Typically the system does not stream mavlink messages after boot failure, in this case check the error messages that are printed on the system console.
+
+The following example shows how to start custom applications:
+  * Create a file on the SD card `etc/extras.txt` with this content:
+    ```
+    custom_app start
+    ```
+  * A command can be made optional by gating it with the `set +e` and `set -e` commands:
+    ```
+    set +e
+    optional_app start      # Will not result in boot failure if optional_app is unknown or fails
+    set -e
+
+    mandatory_app start     # Will abort boot if mandatory_app is unknown or fails
+    ```  
+
+#### Starting a custom mixer
+
+By default the system loads the mixer from `/etc/mixers`. 
+If a file with the same name exists in `/fs/microsd/etc/mixers` this file will be loaded instead. This allows to customize the mixer file without the need to recompile the Firmware.
+
+##### Example
+The following example shows how to add a custom aux mixer:
+  * Create a file on the SD card, `etc/mixers/gimbal.aux.mix` with your mixer content.
+  * Then to use it, create an additional file `etc/config.txt` with this content:
+    ```
+    set MIXER_AUX gimbal
+    set PWM_AUX_OUT 1234
+    set PWM_AUX_DISARMED 1500
+    set PWM_AUX_MIN 1000
+    set PWM_AUX_MAX 2000
+    set PWM_AUX_RATE 50
+    ```