-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
docs, samples: add README, statemachine docs and sample configuration…
… files
- Loading branch information
Showing
58 changed files
with
3,863 additions
and
0 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,28 @@ | ||
## Flasher - Server fleet firmware install automation. | ||
|
||
Flasher is a vendor agnostic tool to automate firmware installats on a server fleet. | ||
|
||
flasher currently supports is Out of band component firmware installs, that is - firmware installs through the server BMC. | ||
|
||
As of this release flasher depends on serverservice for the inventory and firmware information. | ||
|
||
<p align="left"> | ||
<img height="500px" width="450px" src="./docs/flasher-min.png"> | ||
</p> | ||
|
||
|
||
## build | ||
|
||
`make build-linux` | ||
|
||
## run | ||
|
||
see [cheatsheet.md](./docs/cheatsheet.md) | ||
|
||
## Documentation | ||
|
||
Documentation on development and flasher implementation can be found in [docs](./docs/README-devel.md) | ||
|
||
## Supported devices | ||
|
||
For out of band updates, Flasher leverages [bmclib.v2](https://github.com/bmc-toolbox/bmclib/tree/v2) and supports firmware installs on all devices that bmclib.v2 supports. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,56 @@ | ||
# Flasher state machine | ||
|
||
Flasher installs firmware updates and tracks thier progress by executing a set of statemachines, this document describes those statemachines. | ||
|
||
|
||
### Task state machine | ||
|
||
Flasher runs a state machine for each firmware update its installing. | ||
|
||
This *Task statemachine* transitions a task through three states - `queued`, `active`, `failed`, `success`. | ||
|
||
<p align="center"> | ||
<img src="./statemachine/task_sm.svg"> | ||
<br \> | ||
</p> | ||
|
||
|
||
The definition for the *Task statemachine* can be found [here](../internal/statemachine/task.go). | ||
|
||
|
||
In the `active` state a task transitions through the `query`, `plan`, `run` transition handlers. | ||
|
||
The definitions for the *Task statemachine* transition handlers can be found [here](../internal/worker/task_handlers.go). | ||
|
||
Generated documentation for the *Task state-machine* describing the various states and transitions can be found [here](../docs/statemachine/README-task-statemachine.md) | ||
|
||
#### The Query & Plan Transitions | ||
|
||
These are *transition handlers* within the *Task stateachine* which carry out the following steps, | ||
|
||
- Query the device/inventory for current component firmware versions. | ||
- Identify firmware applicable for the device based on current firmware component versions. | ||
- Generate a an *action* plan for each component firmware applicable. | ||
|
||
The actions are generated as "sub-statemachines" which are then executed within the *Task statemachine* as part of the `run` transition handler. | ||
|
||
#### The run Transition and Action sub-statemachine(s) | ||
|
||
Each firmware to be installed is one *Action sub-statemachine* and are executed in the `run` transition handlers. | ||
|
||
The action to install firmware band has its own *Action sub-statemachine* the definition for which can be found [here](../internal/outofband/actions.go) | ||
|
||
|
||
The transition handlers for the *Action sub-statemachine* can be found [here](../internal/outofband/action_handlers.go) | ||
|
||
|
||
The graph below describes the *Action sub-statemachine* flow | ||
|
||
<p align="center"> | ||
<img src="./statemachine/action_sm.svg"> | ||
<br \> | ||
</p> | ||
|
||
|
||
Generated documentation for the *Action state-machine* describing the various states and transitions can be found [here](../docs/statemachine/README-action-statemachine.md) | ||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,39 @@ | ||
# get flasher task attribute for device | ||
|
||
``` | ||
flasher get task --device-id f0c8e4ac-5cce-4370-93ff-bd3196fd3b9e \ | ||
--config ./samples/flasher-client.yaml | ||
``` | ||
|
||
# set device for firmware install - force to skip installed component version checks. | ||
``` | ||
flasher set install-firmware --device-id fa1e8306-b8a6-425b-8f84-fb747d73d399 \ | ||
--inventory-source=serverservice \ | ||
--config ./samples/flasher-client.yaml \ | ||
--force | ||
``` | ||
|
||
# delete firmware install task for device | ||
``` | ||
flasher delete task --device-id f0c8e4ac-5cce-4370-93ff-bd3196fd3b9e \ | ||
--config ./samples/flasher-client.yaml | ||
``` | ||
|
||
# run worker | ||
``` | ||
flasher run worker --inventory-source=serverservice \ | ||
--config ./samples/flasher-worker.yaml \ | ||
--trace | ||
``` | ||
|
||
# export flasher Task action sub-statemachine JSON representation doc | ||
``` | ||
flasher export statemachine --action > ./docs/statemachine/action-statemachine.json | ||
``` | ||
|
||
# export flasher Task statemachine JSON representation doc | ||
``` | ||
flasher export statemachine --task > ./docs/statemachine/task-statemachine.json | ||
``` | ||
|
||
|
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,198 @@ | ||
### Serverservice interaction | ||
|
||
Serverservice is an inventory source for Flasher. | ||
|
||
When Serverservice is enabled through configuration, Flasher queries Serverservice for, | ||
|
||
- Device inventory (includes installed firmware versions). | ||
- Firmware versions data (includes URLs to download the firmware updates and checksums). | ||
- Firmware sets - groups of Firmware versions applicable to a device vendor, model type. | ||
|
||
#### Task server attribute | ||
|
||
In the first release - `v0.1.0`, servers are 'flagged' for updates by an operator, | ||
|
||
Flasher periodically queries Serverservice for servers flagged for firmware updates, | ||
and when it has determined the server passes all the update preconditions, the | ||
update install process is initiated. | ||
|
||
Servers are flagged by setting a server attribute on a server in the namespace, | ||
`sh.hollow.flasher.task`. | ||
|
||
The payload for this is in the form, | ||
|
||
```json | ||
{ | ||
"namespace": "sh.hollow.flasher.task", | ||
// this is one of 'out-of-band', 'in-band' | ||
"method": "out-of-band", | ||
"status": "", | ||
// 0 to 3, 3 being the highest | ||
"priority": 1, | ||
// user requesting this task | ||
"requester": "foo", | ||
... | ||
} | ||
``` | ||
|
||
Once flasher identifies and picks up a task, its status attribute is set, initially to `queued`, | ||
|
||
```json | ||
{ | ||
"namespace": "sh.hollow.flasher.task", | ||
"status": "queued", | ||
"requester": "requester", | ||
"worker": "some-worker-name" | ||
... | ||
} | ||
``` | ||
|
||
The possible states are `queued`, `active`, `sucess`, `failed` | ||
|
||
|
||
#### Firmware sets | ||
|
||
Firmware sets are firmware versions vetted to be working and applicable for a device/component | ||
based on vendor, model attributes, and going ahead other non device specific attributes, like the organization or project the device is part of. | ||
|
||
### Install mode - `PredefinedFirmwareFirmware` | ||
|
||
In server services this would be the firmware versions table. | ||
|
||
Note: firmware sets are ignored in this mode. | ||
|
||
A sample server service flasher install task payload looks like the following, | ||
|
||
```json | ||
{ | ||
"namespace": "sh.hollow.flasher.task", | ||
// user defined install versions | ||
"Firmware": [ | ||
"bmc": { | ||
"version": "1.1", | ||
// these are optional | ||
"preActions": ["resetBMC"], | ||
"postActions": ["resetDevice"], | ||
// | ||
// currenly installed version is not checked - forces downgrades or reinstall of the same firmware | ||
"force": true | ||
} | ||
"bios": { | ||
"version": "2.6.6". | ||
... | ||
} | ||
], | ||
``` | ||
|
||
### Install mode - `PredefinedFirmwareSet` | ||
|
||
In this mode a firmware set name was specified, in the form | ||
|
||
```json | ||
{ | ||
"namespace": "sh.hollow.flasher.task", | ||
// user defined firmware set ID | ||
"firmwareSet": "dell-r6515", | ||
], | ||
... | ||
``` | ||
|
||
Flasher proceeds to lookup the applicable firmware - comparing the versions in the firmware set | ||
and the version installed. | ||
|
||
Once its determined the applicable firmware, flasher populates the `Firmware` field. | ||
|
||
```json | ||
|
||
{ | ||
"namespace": "sh.hollow.flasher.task", | ||
"firmwareSet": "dell-r6515", | ||
// resolved by flasher for this mode. | ||
"Firmware": [ | ||
"bmc": { | ||
"version": "1.1", | ||
} | ||
"bios": { | ||
"version": "2.6.6". | ||
... | ||
} | ||
], | ||
``` | ||
|
||
|
||
### Install mode - `ResolveFirmwareFirmware` | ||
|
||
Flasher looks up firmware sets applicable for the device, components based | ||
on the firmware set labels - `vendor=foo, model=bar`. | ||
|
||
It then compares the installed versions on the components with | ||
the ones available in the set and determines if it should proceed or not. | ||
|
||
In this case the task attribute would initially be, | ||
|
||
```json | ||
{ | ||
"namespace": "sh.hollow.flasher.task", | ||
"status": "queued", | ||
... | ||
} | ||
``` | ||
|
||
once the task has been picked, the `firmwareSet`, `Firmware` attributes are populated | ||
|
||
|
||
```json | ||
|
||
{ | ||
"namespace": "sh.hollow.flasher.task", | ||
// resolved by flasher for this mode. | ||
"firmwareSet": "dell-r6515", | ||
// resolved by flasher for this mode. | ||
"Firmware": [ | ||
"bmc": { | ||
"version": "1.1", | ||
} | ||
"bios": { | ||
"version": "2.6.6". | ||
... | ||
} | ||
], | ||
``` | ||
|
||
|
||
### Flasher states and metadata | ||
|
||
Flasher is a stateful application, it maintains an internal queue of firmware install tasks, | ||
at some point if its decided that there is going to be multiple flashers | ||
per facility, then this queueing system needs to be external (redis). | ||
|
||
Flasher keeps the state of updates `queued` and `active` by recording the metadata | ||
in Server service as a server attribute in the namespace `sh.hollow.flasher.task`. | ||
|
||
Going ahead, this server attribute is referred as the server firmware install attribute. | ||
|
||
An example of such firmware install metadata as a server attribute is shown below. | ||
|
||
What is not indicated here is that each server attribute contains a created and | ||
last updated timestamp. These timestamps will be used to determine the | ||
'freshness' of an update process. | ||
|
||
```json | ||
{ | ||
"namespace": "sh.hollow.flasher.task", | ||
"firmwareSet": "", | ||
"Firmware": [ | ||
"bmc": { | ||
"version": "1.1", | ||
"updateFileURL": "...", | ||
"updateFileChecksum": "...", | ||
"force": true | ||
} | ||
], | ||
"priority": 0, | ||
"method": "out-of-band", | ||
"status": "active", | ||
"requester": "foobar", | ||
"worker": "flasher-pod-name", | ||
} | ||
``` |
Oops, something went wrong.