Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Smartsim Documentation Refactor #463

Merged
merged 97 commits into from
Mar 13, 2024
Merged
Show file tree
Hide file tree
Changes from 22 commits
Commits
Show all changes
97 commits
Select commit Hold shift + click to select a range
bc0e667
SmartSim Experiment docs (#422)
amandarichardsonn Jan 5, 2024
6a3f963
SmartSim logger documentation (#431)
amandarichardsonn Jan 9, 2024
5a74666
Settings documentation (#429)
amandarichardsonn Jan 12, 2024
9baf6cd
Orchestrator docs (#432)
amandarichardsonn Jan 19, 2024
efcbd74
Model documentation (#436)
amandarichardsonn Jan 25, 2024
365442f
Ensemble docs (#437)
amandarichardsonn Jan 25, 2024
afeb958
Merge branch 'develop' into smartsim-docs-refresh
amandarichardsonn Jan 25, 2024
c96ca0d
update to index
amandarichardsonn Jan 25, 2024
d46586f
push commits 4 review prep
amandarichardsonn Jan 25, 2024
9741b53
Merge branch 'develop' into smartsim-docs-refresh
amandarichardsonn Feb 12, 2024
6300535
small model changes
amandarichardsonn Feb 12, 2024
e1167dc
small fixes
amandarichardsonn Feb 13, 2024
937a908
small changes
amandarichardsonn Feb 13, 2024
335529f
Merge branch 'develop' into smartsim-docs-refresh
amandarichardsonn Feb 13, 2024
9b1b2ca
model doc bug fixes
amandarichardsonn Feb 13, 2024
5f48aad
model func
amandarichardsonn Feb 13, 2024
c3a6670
mdoel updates
amandarichardsonn Feb 13, 2024
440bb23
ensemble doc
amandarichardsonn Feb 14, 2024
8dd109d
pushing completed comments
amandarichardsonn Feb 14, 2024
92398ba
small changes
amandarichardsonn Feb 14, 2024
78af1cd
updates to ensemble doc - no prefix edits
amandarichardsonn Feb 14, 2024
2e98250
Address Ensemble comments
amandarichardsonn Feb 16, 2024
f3675e0
edits made to doc
amandarichardsonn Feb 20, 2024
aeb3664
other comments
amandarichardsonn Feb 20, 2024
7098500
Experiment comments iter 1
amandarichardsonn Feb 21, 2024
8f93244
RunSettings comments iter 1
amandarichardsonn Feb 21, 2024
7406cfe
Model comments iter 1
amandarichardsonn Feb 21, 2024
668350b
edits to logger
amandarichardsonn Feb 22, 2024
b838e9b
pushing edits
amandarichardsonn Feb 24, 2024
a3f5815
Merge branch 'develop' of https://github.com/CrayLabs/SmartSim into s…
amandarichardsonn Feb 26, 2024
c255bb3
make style
amandarichardsonn Feb 26, 2024
2db1e73
Revert "make style"
amandarichardsonn Feb 26, 2024
8c76ccb
edit to readthedocs
amandarichardsonn Mar 4, 2024
12616b4
doc changes
amandarichardsonn Mar 4, 2024
9a37ef8
pandoc version
amandarichardsonn Mar 4, 2024
35497de
test
amandarichardsonn Mar 4, 2024
8a83a2e
testing
amandarichardsonn Mar 4, 2024
d5a0783
smartredis
amandarichardsonn Mar 4, 2024
e51f391
test
amandarichardsonn Mar 4, 2024
9617b5a
readthedocs
amandarichardsonn Mar 5, 2024
fcc6c03
test smartredis build
amandarichardsonn Mar 5, 2024
6ff9519
edit
amandarichardsonn Mar 5, 2024
af8c261
test
amandarichardsonn Mar 5, 2024
6d4a563
read
amandarichardsonn Mar 5, 2024
5a8b929
test
amandarichardsonn Mar 5, 2024
d39d603
test
amandarichardsonn Mar 5, 2024
e58c277
test
amandarichardsonn Mar 5, 2024
2c0265e
test doxygen build
amandarichardsonn Mar 5, 2024
e0f0b27
test link examples
amandarichardsonn Mar 5, 2024
3f33abb
testing experiment import
amandarichardsonn Mar 5, 2024
355e185
test for dashboard failure
amandarichardsonn Mar 5, 2024
ec1cfb1
ignore changelog
amandarichardsonn Mar 5, 2024
4d748b2
change linkcheck_timeout to float
amandarichardsonn Mar 5, 2024
95a2f8f
delete changelog exclude
amandarichardsonn Mar 5, 2024
dec1ae0
issue in build
amandarichardsonn Mar 5, 2024
9583319
test conf fail
amandarichardsonn Mar 5, 2024
1c4d081
build time test
amandarichardsonn Mar 5, 2024
c310590
test changelog
amandarichardsonn Mar 5, 2024
182643a
expand timer check
amandarichardsonn Mar 5, 2024
107f5e3
test
amandarichardsonn Mar 5, 2024
cabdeeb
test
amandarichardsonn Mar 5, 2024
f0bf5df
edits
amandarichardsonn Mar 5, 2024
3fa2bc5
test
amandarichardsonn Mar 5, 2024
d4e5044
test
amandarichardsonn Mar 5, 2024
d72355c
test
amandarichardsonn Mar 5, 2024
16e67ed
changes
amandarichardsonn Mar 5, 2024
040d143
links testing
amandarichardsonn Mar 6, 2024
93d1bce
readthedocs
amandarichardsonn Mar 6, 2024
2ef8422
pushing yaml
amandarichardsonn Mar 6, 2024
a6df500
test
amandarichardsonn Mar 6, 2024
1089c64
test
amandarichardsonn Mar 6, 2024
9215570
test
amandarichardsonn Mar 6, 2024
1e20970
test
amandarichardsonn Mar 6, 2024
4a50b9d
testing links
amandarichardsonn Mar 6, 2024
560dce9
fix links
amandarichardsonn Mar 6, 2024
c4c909d
test PR
amandarichardsonn Mar 6, 2024
b3ce5b5
titles
amandarichardsonn Mar 6, 2024
e00b5d0
pushing make style
amandarichardsonn Mar 6, 2024
cc0b5b0
new line
amandarichardsonn Mar 6, 2024
6f8440c
Revert "pushing make style"
amandarichardsonn Mar 6, 2024
4ea7a0d
edits to the docs
amandarichardsonn Mar 6, 2024
a4b698c
will be removed anyway
amandarichardsonn Mar 6, 2024
d0ccb01
test
amandarichardsonn Mar 7, 2024
8d6c640
jk
amandarichardsonn Mar 7, 2024
e01b4ff
robots.txt
amandarichardsonn Mar 8, 2024
fc7c4e5
test robot
amandarichardsonn Mar 9, 2024
625823b
testing robots.txt path
amandarichardsonn Mar 9, 2024
16ac045
test build
amandarichardsonn Mar 9, 2024
b990927
small changes
amandarichardsonn Mar 9, 2024
e66ff96
new robots.txt file
amandarichardsonn Mar 9, 2024
30c3abc
robots.txt file
amandarichardsonn Mar 11, 2024
7c51188
small fixes
amandarichardsonn Mar 11, 2024
b98d10b
pushing final
amandarichardsonn Mar 12, 2024
075a120
Merge branch 'develop' of https://github.com/CrayLabs/SmartSim into s…
amandarichardsonn Mar 12, 2024
47616ad
changelog update
amandarichardsonn Mar 12, 2024
b701281
rerunning tests
amandarichardsonn Mar 13, 2024
5d7b4a4
attempting to remove readthedocs
amandarichardsonn Mar 13, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 12 additions & 3 deletions doc/api/smartsim_api.rst
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,8 @@ Experiment
:members:


.. _settings-info:

Settings
========

Expand Down Expand Up @@ -388,12 +390,14 @@ Orchestrator
Orchestrator
------------

.. _orchestrator_api:

.. autoclass:: Orchestrator
:members:
:inherited-members:
:undoc-members:

.. _model_api:

Model
=====
Expand All @@ -417,17 +421,17 @@ Model
Model.disable_key_prefixing
Model.query_key_prefixing

Model
-----

.. autoclass:: Model
:members:
:show-inheritance:
:inherited-members:

.. _ensemble_api:

Ensemble
========


.. currentmodule:: smartsim.entity.ensemble

.. autosummary::
Expand All @@ -443,6 +447,11 @@ Ensemble
Ensemble.query_key_prefixing
Ensemble.register_incoming_entity

Ensemble
--------

.. _ensemble_api:

.. autoclass:: Ensemble
:members:
:show-inheritance:
Expand Down
113 changes: 113 additions & 0 deletions doc/batch_settings.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,113 @@
.. _batch_settings_doc:
**************
Batch Settings
**************
========
Overview
========
SmartSim provides functionality to launch entities (``Model`` or ``Ensemble``)
as batch jobs supported by the ``BatchSettings`` base class. While the ``BatchSettings`` base
class is not intended for direct use by users, its derived child classes offer batch
launching capabilities tailored for specific workload managers (WLMs). Each SmartSim
`launcher` interfaces with a ``BatchSettings`` subclass specific to a system's WLM:

- The Slurm `launcher` supports:
- :ref:`SbatchSettings<sbatch_api>`
- The PBS Pro `launcher` supports:
- :ref:`QsubBatchSettings<qsub_api>`
- The LSF `launcher` supports:
- :ref:`BsubBatchSettings<bsub_api>`

.. note::
The local `launcher` does not support batch jobs.

After creating a ``BatchSettings`` instance, users gain access to the attributes and methods
of the associated child class, providing them with the ability to further configure the batch
settings for jobs.

In the following :ref:`Examples<batch_settings_ex>` subsection, we demonstrate the initialization
and configuration of a batch settings object for each supported SmartSim `launcher`.

.. _batch_settings_ex:
========
Examples
========
A ``BatchSettings`` child class is created using the ``Experiment.create_batch_settings()``
factory method. When the user initializes the ``Experiment`` at the beginning of the Python driver script,
they may specify a `launcher` argument. SmartSim will then register or detect the `launcher` and return the
corresponding supported child class when ``Experiment.create_batch_settings()`` is called. This
design allows SmartSim driver scripts utilizing ``BatchSettings`` to be portable between systems,
requiring only a change in the specified `launcher` during ``Experiment`` initialization.

Below are examples of how to initialize a ``BatchSettings`` object per `launcher`.

.. tabs::

.. group-tab:: Slurm
To instantiate the ``SbatchSettings`` object, which interfaces with the Slurm job scheduler, specify
`launcher="slurm"` when initializing the ``Experiment``. Upon calling ``create_batch_settings()``,
amandarichardsonn marked this conversation as resolved.
Show resolved Hide resolved
SmartSim will detect the job scheduler and return the appropriate batch settings object.

.. code-block:: python

from smartsim import Experiment

# Initialize the experiment and provide launcher Slurm
exp = Experiment("name-of-experiment", launcher="slurm")

# Initialize a SbatchSettings object
sbatch_settings = exp.create_batch_settings(nodes=1, time="10:00:00", batch_args={"ntasks": 1})
amandarichardsonn marked this conversation as resolved.
Show resolved Hide resolved
# Set the account for the slurm batch job
sbatch_settings.set_account("12345-Cray")
# Set the partition for the slurm batch job
sbatch_settings.set_queue("default")

The initialized ``SbatchSettings`` instance can now be passed to a SmartSim entity
(``Model`` or ``Ensemble``) via the `batch_settings` argument in ``create_batch_settings()``.

.. group-tab:: PBS Pro
To instantiate the ``QsubBatchSettings`` object, which interfaces with the PBS Pro job scheduler, specify
`launcher="pbs"` when initializing the ``Experiment``. Upon calling ``create_batch_settings()``,
SmartSim will detect the job scheduler and return the appropriate batch settings object.

.. code-block:: python

from smartsim import Experiment

# Initialize the experiment and provide launcher PBS Pro
exp = Experiment("name-of-experiment", launcher="pbs")

# Initialize a QsubBatchSettings object
qsub_batch_settings = exp.create_batch_settings(nodes=1, time="10:00:00", batch_args={"ntasks": 1})
amandarichardsonn marked this conversation as resolved.
Show resolved Hide resolved
# Set the account for the PBS Pro batch job
qsub_batch_settings.set_account("12345-Cray")
# Set the partition for the PBS Pro batch job
qsub_batch_settings.set_queue("default")

The initialized ``QsubBatchSettings`` instance can now be passed to a SmartSim entity
(``Model`` or ``Ensemble``) via the `batch_settings` argument in ``create_batch_settings()``.

.. group-tab:: LSF
To instantiate the ``BsubBatchSettings`` object, which interfaces with the LSF job scheduler, specify
`launcher="lsf"` when initializing the ``Experiment``. Upon calling ``create_batch_settings()``,
SmartSim will detect the job scheduler and return the appropriate batch settings object.

.. code-block:: python

from smartsim import Experiment

# Initialize the experiment and provide launcher LSF
exp = Experiment("name-of-experiment", launcher="lsf")

# Initialize a BsubBatchSettings object
bsub_batch_settings = exp.create_batch_settings(nodes=1, time="10:00:00", batch_args={"ntasks": 1})
# Set the account for the lsf batch job
bsub_batch_settings.set_account("12345-Cray")
# Set the partition for the lsf batch job
bsub_batch_settings.set_queue("default")

The initialized ``BsubBatchSettings`` instance can now be passed to a SmartSim entity
(``Model`` or ``Ensemble``) via the `batch_settings` argument in ``create_batch_settings()``.

.. warning::
Note that initialization values provided (e.g., `nodes`, `time`, etc) will overwrite the same arguments in `batch_args` if present.
3 changes: 2 additions & 1 deletion doc/conf.py
Original file line number Diff line number Diff line change
Expand Up @@ -52,7 +52,8 @@
'breathe',
'nbsphinx',
'sphinx_copybutton',
'sphinx_tabs.tabs'
'sphinx_tabs.tabs',
'sphinx_panels'
]

suppress_warnings = ['autosectionlabel']
Expand Down
Loading