Add a topic section on performance

docs/source/howto/installation.rst

@@ -396,7 +396,7 @@ Make sure the daemon is running (``verdi daemon start``) and then execute the sc
 
 This will launch 100 ``ArithmeticAddCalculation`` jobs on the localhost and record the time until completion.
 
-Below is the output for running 100 ``ArithmeticAddCalculation``s with a single daemon worker on one core of an AMD Ryzen 5 3600 6-Core processor, using AiiDA v1.6.9, and RabbitMQ and PostgreSQL running on the same machine:
+Below is the output for running 100 ``ArithmeticAddCalculation``'s with a single daemon worker on one core of an AMD Ryzen 5 3600 6-Core processor, using AiiDA v1.6.9, and RabbitMQ and PostgreSQL running on the same machine:
 
 .. code:: console
 
@@ -431,6 +431,7 @@ This rate can be sped up by adding more daemon workers:
     ========== ======================= ==========================
 
 A decent setup for AiiDA (for comparable hardware as described above) should show a performance of roughly 2 processes per second for a single worker.
+For more information on AiiDA's performance and how to optimize it, please refer to the :ref:`the topic on performance<topics:performance>`.
 
 .. note::
 

docs/source/topics/index.rst

@@ -15,6 +15,7 @@ Topics
    plugins
    schedulers
    transport
+   performance
 
 .. todo::
 

docs/source/topics/performance.rst

@@ -0,0 +1,67 @@
+.. _topics:performance:
+
+***********
+Performance
+***********
+
+The performance of AiiDA depends on many factors:
+
+* the hardware that AiiDA is running on
+* the environment in which AiiDA is installed
+* how the services for AiiDA are configured (the database, message broker, filesystem, etc.)
+* the codes and their plugins that are being run.
+
+This section gives an overview of how each of these factors influence the overall performance of AiiDA and how it can be optimized.
+
+
+.. _topics:performance:hardware:
+
+Hardware
+========
+
+The bulk of AiiDA's workload is typically carried by the daemon and its workers.
+The performance is typically limited by the computing power of the machine on which AiiDA is running.
+
+Each worker is a separate Python process that takes care of executing the AiiDA processes that are submitted.
+AiiDA was designed to allow to increase the throughput by adding more daemon workers that can work independently in parallel.
+A rule of thumb is to not have more workers than the number of cores of the machine's CPU on which AiiDA is running.
+If more workers are added, they will have to start sharing and swapping resources and the performance scaling will degrade.
+
+
+.. _topics:performance:environment:
+
+Environment
+===========
+
+To be added.
+
+
+.. _topics:performance:services:
+
+Services
+========
+
+For the default setup, AiiDA essentially has three services that influence its performance:
+
+* PostgreSQL (the database in which the provenance graph is stored)
+* RabbitMQ (the message broker that the daemon workers use to communicate)
+* Filesystem (files are stored by AiiDA in the file repository on a filesytem)
+
+For the simplest installations, the PostgreSQL and RabbitMQ services are typically running on the same machine as AiiDA itself.
+Although this means that a part of the machine's resources is not available for AiiDA itself and its daemon, the latency for AiiDA to communicate with the services is minimal.
+
+It is possible to configure an AiiDA profile to use services that are running on a different machine and can be reached over a network.
+However, this will typically affect the performance negatively as now each time a connection needs to be made to a service, the latency of the network is incurred.
+
+
+.. _topics:performance:plugins:
+
+Plugins
+=======
+
+One of AiiDA's strengths is its plugin system, which allows it capabilities to be customized in a variety of ways.
+However, this flexibility also means that the performance of AiiDA can be affected significantly by the implementation of the plugins.
+For example, a `CalcJob` plugin determines which files are transferred from and to the computing resources.
+If the plugin needs to transfer and store large amounts of data, this will affect the process throughput of the daemon workers.
+Likewise, if a `Parser` plugin performs heavy numerical computations to parse the retrieved data, this will slow down the workers' throughput.
+In order to optimize the process throughput, plugins should try to minize heavy computations and the transfer of lots of unnecessary data.