Fleetspeak is a client-server framework for communicating with a fleet of machines. This document summarizes the various components, providing an overview of what bits serve what purpose.
Fleetspeak identifies clients using a fingerprint of a public key, much like
openssl keys. Specifically, client creates a keypair on startup, or when asked
to rekey. The Fleetspeak
common.ClientID
is the first 8 bytes of a sha256 hash of the public part of this key. It is the
Server Communicator's responsibility to verify the identity of
clients. The recommended approach is to communicate over TLS using the client's
Fleetspeak key as TLS client identification. The https.Communicator
source
has an example of this.
At its core, Fleetspeak forwards
fleetspeak.Message
protocol buffers. The comments in the .proto file should be considered
definitive, but the most important fields to understand are destination,
message_type and data. Essentially, destination field tells Fleetspeak
where to forward the message to. The message_type and data fields are
forwarded verbatim, and when integrating a service a developer should set these
fields according to their needs.
The message_type is meant to be used both for dispatching within a service,
and for reporting. So for example if your integration has three basic types of
messages to pass, you might indicate them with three different message type
strings. By doing this you make it possible to query Fleetspeak for the number
of messages of each type.
A Fleetspeak Server middleware process written in go. To start a server, you
instantiate a
server.Server,
providing the components needed for your particular installation.
The
miniserver
binary from the demo directory defines a Fleetspeak Server in a way suitable for
demonstrations and small installations.
When run, the miniserver binary binds two ports, and accepts bind addresses for
each from the command line. The --https_addr flag determines the interface and
port that clients will connect to. It needs to be open to the internet, or at
least to all Fleetspeak client machines. See Server
Communicator, below.
The --admin_addr flag determines the interface and port that the
Administrative Interface listens on. The miniserver
binary does not perform any authentication of administrative requests. Therefore
access to this port needs to be limited to trusted processes.
The miniserver process stores all of its state in an SQLite version 3
database. The location of this database file is set by the flag
--database_path. This file will be created if missing. After running
miniserver, you can examine the system state using, e.g., the sqlite3 command
and package on Debian systems. See Datastore below.
The Administrative Interface is a GRPC server which provides access to Fleetspeak state for administrative and integration purposes. It is implemented separately from the Fleetspeak Server and can be run from the same or a different process, so long as it is connected to the same Datastore
The
db.Store
interface defines the persistent state held by a Fleetspeak Server. If a
Fleetspeak installation has multiple server processes, they should run the same
Datastore implementation providing a view of the same database. Currently we
provide two Datastore implementations.
At a minimum, any Datastore implementation should pass the tests implemented by
the dbtesting
package.
The
sqlite.Datastore
implements
db.Store
backed by an SQLite version 3 database. This implementation is used extensively
in our unit tests, and its design choices focus on simplicity, safety and
ease-of-use rather than performance. It is meant for testing, local
demonstrations, and single server installations with a small number of clients.
It does not support multiple servers processes - the SQLite database file should
only be opened by one instance of sqlite.Datastore at a time.
The
mysql.Datastore
implements
db.Store
backed by a mysql database. Using recent versions of the
go-sql-driver/mysql driver it passes
dbtesting,
but performance testing and database optimization is still in progress.
Every server requires a
Communicator
component which handles communication with clients. This component defines how
the server communicates with fleetspeak machines and should agree on protocol
with a corresponding Client Communicator.
The quintessential example is
https.Communicator
and at this point it is the expected implementation for essentially all
installations. By adjusting the passed in net.Listener parameter, it should be
possible to include any needed additional support for load balancing or other
reverse proxying.
When messages arrive on a Fleetspeak server, they must be addressed to a
particular service - the destination field as described in
Messages. A Fleetspeak server runs one or more Services in order
to handle these messages. By configuring additional independent services a
single Fleetspeak installation can handle messages for independent purposes.
A service is configured by a
fleetspeak.server.ServiceConfig
protocol buffer. Besides the name, used to address the service, the most important parameter
is factory. This string is used to look up a
service.Factory,
which determines what code to use to process incoming messages.
As a particularly simple example, the fleetspeak demo directory includes a
messagesaver
package which defines a service which simply saves all received messages to text
files.
The
grpcservice
package provides a service.Factory which makes a GRPC call
for each delivered message, in this way handing the message off to another
process.
It expects configuration parameters to be provided in a
fleetspeak.grpcservice.Config
protocol buffer, and the target GRPC server must implement the
fleetspeak.grpcservice.Processor GRPC interface.
In addition to the factory, the grpcservice package also exports a concrete type which can be used to derive a GRPC-based service with more control over how and when it dials a new GRPC target.
The Fleetspeak Client is a small process which runs on an endpoint and
communicates with a Fleetspeak Server. Much like the server, it consists of a
base
client.Client
along with a collection of components, and individual installations may adjust
these components according to specific needs.
The
miniclient
binary from the demo directory defines a Fleetspeak Client in a general and
flexible way. Individual installations may want to define their own client
binary in order to hardcode security critical parameters (for example, before
signing it as a trusted binary) and otherwise adjust client behavior.
A Fleetspeak client must be provided with a list of TLS root certificates. The
client will then only communicate with a server if it presents one of these
certs, or a cert chaining to one of these certs. The --trusted_cert_file flag
is required by miniclient to set this list.
A Fleetspeek client must also be provided with the server (host and port) to
connect to. The client will require that the presented TLS server be valid for
this host. The --server flag is required by miniclient to set this server.
In addition, a Fleetspeak client needs a public key that it will use to check
the signature of externally provided service configurations. The implications of
this are discussed in more detail below. The --deployment_key_file flag is
required by miniclient to configure this.
To build a binary that can only be used by a specific Fleetspeak installation,
we recommend creating binary which hardcodes the configuration parameters set by
--trusted_cert_file, --server, and --deployment_key_file. Specifically, we
strongly recommend this if you will sign, whitelist a hash, or otherwise mark
the Fleetspeak client binary as trusted.
In addition to the security critical parameters described in the previous
section, a Fleetspeak client normally requires a configuration directory to
store its private key and to look for additional configuration. See the comments
on config.Configuration for details.
Every client requires a
Communicator
component which handles communication with the server. This component defines how
the client communicates with the Fleetspeak server and should agree on protocol
with a corresponding Server Communicator.
The quintessential example is
https.Communicator
and at this point it is the expected implementation for essentially all
installations. By adjusting the DialContext attribute before starting the
Fleetspeak client, it should be possible to include any needed additional
support for specialized tunneling or proxying.
When messages arrive on a Fleetspeak client, they must be addressed to a
particular service - the destination field as described in
Messages. A Fleetspeak client runs one or more Services in order
to handle these messages, and create message to send to the server. By
configuring additional independent services a single Fleetspeak client process
can handle messages for independent purposes.
A service is typically configured by dropping a
fleetspeak.SignedClientServiceConfig
into <config_path>/services/. Besides the name, used to address the service,
the most important parameter is factory. This string is used to look up a
service.Factory,
which determines what code to use to process incoming messages.
This dropped configuration file must be signed with the deployment key. Therefore by controlling access to the private part of the deployment key, and hardcoding the public exponent into the client binary, allows an installation to maintain strong control over what a particular client binary is capable of.
The
stdinservice.Factory
runs a binary with flags and standard input provided by a message, and returns
the output it produces. The service configuration determines what binary to
run. Every message received by the service causes and execution of the
binary. The configuration used in the demo
directory
sets up services based on this for the ls and cat binaries.
The
daemonservice.Factory
handles the use case in which you want Fleetspeak to run a sub-process and
send/receive messages to/from it. This gives full control over what
is sent to the server, but requires more integration. The sub-process should use
the daemonservice client
library
(currently available for go and python) to communicate through Fleetspeak.
The
socketservice.Factory
handles the use case in which you want Fleetspeak and some separately running
process to find each other and communicate using a local filesystem path,
e.g. through a UNIX domain socket. Much like Daemonservice
this also gives full control over what is sent to the server and requires some
integration. The sister process should use the socketservice client
library
to communicate through Fleetspeak.