Added first version of Readme

README.adoc

@@ -0,0 +1,99 @@
+= Stackable Agent
+
+== Purpose
+The Stackable Agent is an alternative to the Kubernetes Kubelet that executes Pods not in containers but using systemd as its backend.
+It is implemented in Rust as a https://github.com/deislabs/krustlet[Krustlet] provider.
+
+In principle this behavior is similar to that of the the https://github.com/virtual-kubelet/virtual-kubelet[virtual kubelet] project.
+More specifically the functionality of this agent closely resembles that of https://github.com/virtual-kubelet/systemk[systemk].
+The reason why both projects currently exist is simply that systemk was not around when we started Stackable and so we had to go and build our own.
+
+WARNING: We are currently evaluating whether or not we should instead use systemk and contribute to that project instead of building our own agent under https://github.com/stackabletech/agent/issues/42[issue #42]
+
+The agent registers itself as a node with a kube-apiserver and will be considered by the Kubernetes scheduler for workloads (pods).
+To avoid _normal_ Kubernetes pods being scheduled on the Stackable agent (it would not know what to do with these) the agent assigns the following taints to its node object:
+
+|===
+|Taint |Type|Value
+
+|kubernetes.io/arch
+|NoSchedule
+|stackable-linux
+
+|kubernetes.io/arch
+|NoExecute
+|stackable-linux
+|===
+
+These taints _suggest_ to the Kubernetes scheduler that only pods with matching tolerations should be scheduled on this node.
+
+== Installation
+=== Build from source
+Building from source is fairly straightforward if you have the rust toolchain installed, our CI chain tests against the latest stable version at the time the checks are run.
+If you need to install this, generally the recommended way is to use https://rustup.rs/[rustup].
+
+After rust is installed simply run
+
+    cargo build
+
+To create the binary file in _target/debug_.
+
+=== Download binary
+We do not at this time provide pre-compiled binaries, as we are still in the process of setting up the build jobs to create these.
+This readme will be updated as soon as binary downloads are available!
+
+=== OS Packages
+We do not at this time provide OS packages, that is due to change in the near future and this readme will be updated accordingly!
+
+== Configuration
+
+=== Command Line Parameters
+The agent accepts the following command line parameters:
+
+include::documentation/commandline_args.adoc[]
+
+=== Config File
+In addition to directly specifying them on the command line, the agent allows specifying a config file via the environment variable `CONFIG_FILE`. Values specified in the file will have to adhere to the format `--parameter=value`.
+
+This file can contain all command line parameters and will be parsed before the actual command line.
+For parameters that are present in the file and on the command line, the command line will take precedence, unless it is a parameter that can be specified multiple times, in which case parameters from both, file and commandline, will be merged.
+
+.Example config file
+    --package-directory=/opt/stackable/agent/work/packages
+    --config-directory=/etc/stackable/agent
+    --server-cert-file=/etc/stackable/agent/secure/cert.crt
+    --server-key-file=/etc/stackable/agent/secure/key.key
+
+=== Kubernetes Config
+The agent uses the default way of looking for a kube-apiserver, so if your system is already set up to connect to Kubernetes with kubectl you should be good to go right of the bat.
+
+The default location for the Kubernetes client config is `~/.kube/config`, if you want to change this location you can override this via the `KUBECONFIG` environment variable.
+
+    export KUBECONFIG=/etc/stackable/agent/kubeconfig
+
+
+=== Certificates
+The agent requires a keypair and signed certificate to start a webserver which can be used to handle callbacks.
+If these are not specified on the commandline, the agent will create a keypair, upload a certificate signing request to Kubernetes and wait for the certificate before continuing.
+These steps require a certificate manager to be set up and running in your Kubernetes cluster, which may or may not be the case.
+
+You can also manually create these files and specify them on the command line.
+The following example shows how to create these files using https://github.com/OpenVPN/easy-rsa[easy-rsa], but this can be done in any number of different ways as well.
+
+    ./easyrsa init-pki
+    ./easyrsa build-ca
+    ./easyrsa gen-req krustlet1
+    ./easyrsa import-req pki/reqs/krustlet1.req krustlet1-req
+    ./easyrsa sign-req serverClient krustlet1-req
+    # Convert key to pksc8 format
+    openssl pkcs8 -topk8 -inform PEM -outform PEM -nocrypt -in pki/private/krustlet1.key -out pkcs8.key
+
+== Contributing
+The agent is developed as an open source tool and we absolutely welcome any and all contributions!
+Don't hesitate to drop us a line at info@stackable.de or reach out directly to any of our committers / contributors.
+
+We will run a bi-weekly dev call during which we will discuss current development, issues, our children and the general state of the world.
+Please feel free to drop in if you have the time!
+
+Dial in info coming soon!
+

documentation/commandline_args.adoc

@@ -0,0 +1,166 @@
+
+
+
+=== tag
+
+*Default value*: No default value
+
+*Required*: false
+
+*Multiple values:* true
+
+
+A "key=value" pair that should be assigned to this agent as tag. This can be specified multiple times to assign additional tags.
+
+Tags are the main way of identifying nodes to assign services to later on.
+
+
+=== server-key-file
+
+*Default value*: No default value
+
+*Required*: false
+
+*Multiple values:* false
+
+
+Private key file (in PKCS8 format) to use for the local webserver the Krustlet starts.
+
+
+=== log-directory
+
+*Default value*: /opt/stackable/logs
+
+*Required*: false
+
+*Multiple values:* false
+
+
+This directory will serve as starting point for all log files which this service creates.
+Every service will get its own subdirectory created within this directory.
+Anything that is then specified in the log4j config or similar files will be resolved relatively to this directory.
+
+The agent will need full access to this directory and tries to create it if it does not exist.
+
+
+=== hostname
+
+*Default value*: No default value
+
+*Required*: false
+
+*Multiple values:* false
+
+
+The hostname to register the node under in Kubernetes - defaults to system hostname.
+
+
+=== server-cert-file
+
+*Default value*: No default value
+
+*Required*: false
+
+*Multiple values:* false
+
+
+The certificate file for the local webserver which the Krustlet starts.
+
+
+=== no-config
+
+*Default value*: No default value
+
+*Required*: false
+
+*Multiple values:* false
+
+
+If this option is specified, any file referenced in AGENT_CONF environment variable will be ignored.
+
+
+=== bootstrap-file
+
+*Default value*: /etc/kubernetes/bootstrap-kubelet.conf
+
+*Required*: false
+
+*Multiple values:* false
+
+
+The bootstrap file to use in case Kubernetes bootstraping is used to add the agent.
+
+
+=== server-bind-ip
+
+*Default value*: No default value
+
+*Required*: false
+
+*Multiple values:* false
+
+
+The local IP to register as the node's ip with the apiserver. Will be automatically set to the first address of the first non-loopback interface if not specified.
+
+
+=== data-directory
+
+*Default value*: /var/stackable/agent/data
+
+*Required*: false
+
+*Multiple values:* false
+
+
+The directory where the stackable agent should keep its working data.
+
+
+=== config-directory
+
+*Default value*: /opt/stackable/config
+
+*Required*: false
+
+*Multiple values:* false
+
+
+This directory will serve as starting point for all log files which this service creates.
+
+Every service will get its own subdirectories created within this directory - for every service start a
+new subdirectory will be created to show a full history of configuration that was used for this service.
+
+ConfigMaps which are specified in the pod that describes this service will be created relative to these run
+directories - unless the mounts specify an absolute path, in which case it is allowed to break out of this directory.
+
+WARNING: This allows anybody who can specify pods more or less full access to the file system on the machine running the agent!
+
+The agent will need full access to this directory and tries to create it if it does not exist.
+
+
+=== server-port
+
+*Default value*: 3000
+
+*Required*: false
+
+*Multiple values:* false
+
+
+Port to listen on for callbacks.
+
+
+=== package-directory
+
+*Default value*: /opt/stackable/packages
+
+*Required*: false
+
+*Multiple values:* false
+
+
+This directory will serve as starting point for packages that are needed by pods assigned to this node.\n Packages will be downloaded into the "_download" folder at the top level of this folder as archives and remain there for potential future use.
+
+Archives will the be extracted directly into this folder in subdirectories following the naming
+scheme of "productname-productversion".
+
+The agent will need full access to this directory and tries to create it if it does not exist.

src/main.rs → src/bin/agent.rs

@@ -8,11 +8,8 @@ use kubelet::Kubelet;
 use log::{info, warn};
 use stackable_config::ConfigBuilder;
 
-use crate::agentconfig::AgentConfig;
-use crate::provider::StackableProvider;
-
-mod agentconfig;
-mod provider;
+use stackable_agent::config::AgentConfig;
+use stackable_agent::provider::StackableProvider;
 
 #[tokio::main(threaded_scheduler)]
 async fn main() -> anyhow::Result<()> {

src/bin/generate_doc.rs

@@ -0,0 +1,20 @@
+/// This is a helper binary which generates the file _documentation/commandline_args.adoc_ which
+/// contains documentation of the available command line options for the agent binary.
+///
+/// It gets the content by calling [`stackable_agent::config::AgentConfig::get_documentation()`]
+///
+/// # Panics
+/// This will panic if an error occurs when trying to write the file.
+
+fn main() {
+    use stackable_agent::config::AgentConfig;
+    use std::fs;
+
+    let target_file = "documentation/commandline_args.adoc";
+    fs::write(target_file, AgentConfig::get_documentation()).unwrap_or_else(|err| {
+        panic!(
+            "Could not write documentation to [{}]: {}",
+            target_file, err
+        )
+    });
+}

src/agentconfig.rs → src/config/mod.rs

@@ -10,7 +10,7 @@ use pnet::datalink;
 use stackable_config::{ConfigOption, Configurable, Configuration};
 use thiserror::Error;
 
-use crate::agentconfig::AgentConfigError::{ArgumentParseError, WrongArgumentCount};
+use crate::config::AgentConfigError::{ArgumentParseError, WrongArgumentCount};
 
 #[derive(Error, Debug)]
 pub enum AgentConfigError {
@@ -278,6 +278,28 @@ impl AgentConfig {
             .into_string()
             .map_err(|_| anyhow::anyhow!("invalid utf-8 hostname string"))
     }
+
+    pub fn get_documentation() -> String {
+        let mut doc_string = String::new();
+        for option in AgentConfig::get_options() {
+            doc_string.push_str(&format!("\n\n\n=== {}\n\n", option.name));
+            doc_string.push_str(&format!(
+                "*Default value*: `{}`\n\n",
+                option.default.unwrap_or("No default value")
+            ));
+            doc_string.push_str(&format!("*Required*: {}\n\n", option.required));
+            doc_string.push_str(&format!("*Multiple values:* {}\n\n\n", option.list));
+
+            // We have not yet specified a documentation string for all options, as an interim
+            // solution we use the help string for the docs, if no proper doc has been written yet.
+            if option.documentation.is_empty() {
+                doc_string.push_str(&option.help);
+            } else {
+                doc_string.push_str(&option.documentation);
+            }
+        }
+        doc_string
+    }
 }
 
 impl Configurable for AgentConfig {

src/lib.rs

@@ -0,0 +1,2 @@
+pub mod config;
+pub mod provider;