Skip to content

Latest commit

 

History

History
169 lines (120 loc) · 5.24 KB

maintenance_and_operations.asciidoc

File metadata and controls

169 lines (120 loc) · 5.24 KB
Table of Contents

Putting nodes in maintenance

When doing maintenance on a cluster, it is always preferred to set the targeted nodes in a maintenance mode so they don’t try to join an existing cluster. You can achieve this using two methods.

Important
 In a three nodes cluster, you can offer service with at least one node.

Using a clean shutdown

If you stop the packetfence-mariadb service properly on a node or if you shutdown your node properly, the cluster will detect this shutdown and continue to operate.

Using --maintenance flag

In order to activate the maintenance mode on a node:

/usr/local/pf/bin/cluster/maintenance --activate

In order to deactivate the maintenance mode on a node:

/usr/local/pf/bin/cluster/maintenance --deactivate

In order to see the current maintenance state on a node:

/usr/local/pf/bin/cluster/maintenance

Shutting down a PacketFence Active/Active cluster of three nodes

As PacketFence cluster works in an active/active way, with statefull redundance, the order to stop the servers is not very important.

Note
 The important thing is to start the servers in the opposite order that you will stop them .

Example:

  • Stop order: pf1 → pf2 → pf3

  • Start order: pf3 → pf2 → pf1

Shutdown the servers:

  • Logon to the first server with a SSH terminal

  • Type the following command: shutdown -h now

  • Logon to the next server, with a SSH terminal.

  • Type the following command: ping IP_ADDRESS_OF_THE_FIRST_SERVER

  • Once the server do not responce back, type the following command: shutdown -h now

  • Proceed the same way with the last server.

Bringing up a PacketFence Active/Active cluster of three nodes

We want to bring up the cluster, in the same state it was before the shutdown.

Therefore, we will do the Shutting down procedure, but in reverse.

Bring up the "Last" server stopped

Start the server (pf3 in our example) and wait a couple of minutes and ensure you are able to connect to it using SSH before continuing.

Bring up the next server

Start the next server (pf2 in our example), logon to the SSH terminal.

Once prompted, check the packetFence-mariadb sync with the Master, type the command:

mysql -u root -p
MariaDB> show status like 'wsrep%';
MariaDB [(none)]> show status like "wsrep%";
+------------------------------+-------------------------------------------------------+
| Variable_name                | Value                                                 |
+------------------------------+-------------------------------------------------------+
...
| wsrep_cluster_size           | 2                                                     |
...
| wsrep_connected              | ON                                                    |
...
| wsrep_evs_state              | OPERATIONAL                                           |
...
| wsrep_local_state_comment    | Synced                                                |
...
+------------------------------+-------------------------------------------------------+

Bring up the next server

Once the 2 other servers have synced together, you can start the next server that remains. Logon on terminal and go with a show status like 'wsrep%'; once again.

  • The values must have changed to:

MariaDB [(none)]> show status like "wsrep%";
+------------------------------+-------------------------------------------------------+
| Variable_name                | Value                                                 |
+------------------------------+-------------------------------------------------------+
...
| wsrep_cluster_size           | 3                                                     |
...
| wsrep_evs_state              | OPERATIONAL                                           |
...
| wsrep_local_state_comment    | Synced                                                |
...
+------------------------------+-------------------------------------------------------+
Note
 The wsrep_incoming_addresses will give you the IP addresses of the nodes synced.
Note
 The wsrep_cluster_status will always be Primary, even on the slaves.

Backup procedure

Automatic Backup files

The PacketFence servers have a daily backup done, each night (0:30AM).

If you need to externalize those backups, they are in:

/root/backup

File description:

  • packetfence-exportable-backup-DATE_00h30.tgz is an exportable packetfence backup that contains:

  • packetfence-db-dump-innobackup-DATE_00h30.xbstream.gz are the SQL dump of your MariaDB database.

  • packetfence-config-dump-DATE_00h30.tgz are the dump of the PacketFence files.

Manual backups

In case you need to make a "manual" backup, you can type the following command:

/usr/local/pf/addons/exportable-backup.sh

As the daily automatic backups, you will find the file in:

/root/backup/

Two files will be available, tagged with the Date and Time of your backup.