Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[RLlib; docs] Docs do-over (new API stack): Move "new API stack" info box right below page titles for better visibility. #49921

Merged
merged 2 commits into from
Jan 20, 2025
Merged

[RLlib; docs] Docs do-over (new API stack): Move "new API stack" info box right below page titles for better visibility. #49921

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
594c688
wip
sven1977 Jan 17, 2025
32f24e2
wip
sven1977 Jan 18, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions doc/source/rllib/algorithm-config.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,12 +1,12 @@
.. include:: /_includes/rllib/we_are_hiring.rst

.. include:: /_includes/rllib/new_api_stack.rst

.. _rllib-algo-configuration-docs:

AlgorithmConfig API
===================

.. include:: /_includes/rllib/new_api_stack.rst

RLlib's :py:class:`~ray.rllib.algorithms.algorithm_config.AlgorithmConfig` API is
the auto-validated and type-safe gateway into configuring and building an RLlib
:py:class:`~ray.rllib.algorithms.algorithm.Algorithm`.
Expand Down
6 changes: 2 additions & 4 deletions doc/source/rllib/checkpoints.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,14 +1,12 @@

.. include:: /_includes/rllib/we_are_hiring.rst

.. include:: /_includes/rllib/new_api_stack.rst


.. _rllib-checkpoints-docs:

Checkpointing
=============

.. include:: /_includes/rllib/new_api_stack.rst

RLlib offers a powerful checkpointing system for all its major classes, allowing you to save the
states of :py:class:`~ray.rllib.algorithms.algorithm.Algorithm` instances and their subcomponents
to local disk or cloud storage, and restore previously run experiment states and individual subcomponents.
Expand Down
5 changes: 2 additions & 3 deletions doc/source/rllib/external-envs.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,13 +1,12 @@
.. include:: /_includes/rllib/we_are_hiring.rst

.. include:: /_includes/rllib/new_api_stack.rst

.. _rllib-external-env-setups-doc:


External Environments and Applications
======================================

.. include:: /_includes/rllib/new_api_stack.rst

In many situations, it doesn't make sense for an RL environment to be "stepped" by RLlib.
For example, if you train a policy inside a complex simulator that operates its own execution loop,
like a game engine or a robotics simulation. A natural and user friendly approach is to flip this setup around
Expand Down
5 changes: 2 additions & 3 deletions doc/source/rllib/hierarchical-envs.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,13 +1,12 @@
.. include:: /_includes/rllib/we_are_hiring.rst

.. include:: /_includes/rllib/new_api_stack.rst

.. _rllib-hierarchical-environments-doc:


Hierarchical Environments
=========================

.. include:: /_includes/rllib/new_api_stack.rst

You can implement hierarchical training as a special case of multi-agent RL. For example, consider a two-level hierarchy of policies,
where a top-level policy issues high level tasks that are executed at a finer timescale by one or more low-level policies.
The following timeline shows one step of the top-level policy, which corresponds to four low-level actions:
Expand Down
6 changes: 2 additions & 4 deletions doc/source/rllib/key-concepts.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,14 +1,12 @@

.. include:: /_includes/rllib/we_are_hiring.rst

.. include:: /_includes/rllib/new_api_stack.rst


.. _rllib-key-concepts:

Key concepts
============

.. include:: /_includes/rllib/new_api_stack.rst

To help you get a high-level understanding of how the library works, on this page, you learn about the
key concepts and general architecture of RLlib.

Expand Down
4 changes: 2 additions & 2 deletions doc/source/rllib/multi-agent-envs.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,12 +1,12 @@
.. include:: /_includes/rllib/we_are_hiring.rst

.. include:: /_includes/rllib/new_api_stack.rst

.. _rllib-multi-agent-environments-doc:

Multi-Agent Environments
========================

.. include:: /_includes/rllib/new_api_stack.rst

In a multi-agent environment, multiple "agents" act simultaneously, in a turn-based
sequence, or through an arbitrary combination of both.

Expand Down
5 changes: 2 additions & 3 deletions doc/source/rllib/new-api-stack-migration-guide.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,8 +1,5 @@
.. include:: /_includes/rllib/we_are_hiring.rst

.. include:: /_includes/rllib/new_api_stack.rst


.. _rllib-new-api-stack-migration-guide:

.. testcode::
Expand All @@ -15,6 +12,8 @@
New API stack migration guide
=============================

.. include:: /_includes/rllib/new_api_stack.rst

This page explains, step by step, how to convert and translate your existing old API stack
RLlib classes and code to RLlib's new API stack.

Expand Down
5 changes: 2 additions & 3 deletions doc/source/rllib/package_ref/algorithm-config.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,13 +1,12 @@
.. include:: /_includes/rllib/we_are_hiring.rst

.. include:: /_includes/rllib/new_api_stack.rst

.. _algorithm-config-reference-docs:


Algorithm Configuration API
===========================

.. include:: /_includes/rllib/new_api_stack.rst

.. currentmodule:: ray.rllib.algorithms.algorithm_config

Constructor
Expand Down
4 changes: 2 additions & 2 deletions doc/source/rllib/package_ref/algorithm.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,12 +1,12 @@
.. include:: /_includes/rllib/we_are_hiring.rst

.. include:: /_includes/rllib/new_api_stack.rst

.. _algorithm-reference-docs:

Algorithms
==========

.. include:: /_includes/rllib/new_api_stack.rst

The :py:class:`~ray.rllib.algorithms.algorithm.Algorithm` class is the highest-level API in RLlib responsible for **WHEN** and **WHAT** of RL algorithms.
Things like **WHEN** should we sample the algorithm, **WHEN** should we perform a neural network update, and so on.
The **HOW** will be delegated to components such as ``RolloutWorker``, etc..
Expand Down
4 changes: 3 additions & 1 deletion doc/source/rllib/package_ref/callback.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,10 +1,12 @@
.. include:: /_includes/rllib/new_api_stack.rst
.. include:: /_includes/rllib/we_are_hiring.rst

.. _rllib-callback-reference-docs:

Callback APIs
=============

.. include:: /_includes/rllib/new_api_stack.rst

Callback APIs enable you to inject code into an experiment, an Algorithm,
and the subcomponents of an Algorithm.

Expand Down
5 changes: 2 additions & 3 deletions doc/source/rllib/package_ref/distributions.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,13 +1,12 @@
.. include:: /_includes/rllib/we_are_hiring.rst

.. include:: /_includes/rllib/new_api_stack.rst


.. _rllib-distributions-reference-docs:

Distribution API
================

.. include:: /_includes/rllib/new_api_stack.rst

.. currentmodule:: ray.rllib.models.distributions

Base Distribution class
Expand Down
5 changes: 2 additions & 3 deletions doc/source/rllib/package_ref/env.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,13 +1,12 @@

.. include:: /_includes/rllib/we_are_hiring.rst

.. include:: /_includes/rllib/new_api_stack.rst

.. _env-reference-docs:

Environments
============

.. include:: /_includes/rllib/new_api_stack.rst

RLlib mainly supports the `Farama gymnasium API <https://gymnasium.farama.org/>`__ for
single-agent environments, and RLlib's own :py:class:`~ray.rllib.env.multi_agent_env.MultiAgentEnv`
API for multi-agent setups.
Expand Down
5 changes: 2 additions & 3 deletions doc/source/rllib/package_ref/env/env_runner.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,13 +1,12 @@

.. include:: /_includes/rllib/we_are_hiring.rst

.. include:: /_includes/rllib/new_api_stack.rst

.. _env-runner-reference-docs:

EnvRunner API
=============

.. include:: /_includes/rllib/new_api_stack.rst

rllib.env.env_runner.EnvRunner
------------------------------

Expand Down
5 changes: 2 additions & 3 deletions doc/source/rllib/package_ref/env/multi_agent_env.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,13 +1,12 @@

.. include:: /_includes/rllib/we_are_hiring.rst

.. include:: /_includes/rllib/new_api_stack.rst

.. _multi-agent-env-reference-docs:

MultiAgentEnv API
=================

.. include:: /_includes/rllib/new_api_stack.rst

rllib.env.multi_agent_env.MultiAgentEnv
---------------------------------------

Expand Down
4 changes: 2 additions & 2 deletions doc/source/rllib/package_ref/env/multi_agent_env_runner.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,12 +1,12 @@
.. include:: /_includes/rllib/we_are_hiring.rst

.. include:: /_includes/rllib/new_api_stack.rst

.. _multi-agent-env-runner-reference-docs:

MultiAgentEnvRunner API
=======================

.. include:: /_includes/rllib/new_api_stack.rst

rllib.env.multi_agent_env_runner.MultiAgentEnvRunner
----------------------------------------------------

Expand Down
4 changes: 2 additions & 2 deletions doc/source/rllib/package_ref/env/multi_agent_episode.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,12 +1,12 @@
.. include:: /_includes/rllib/we_are_hiring.rst

.. include:: /_includes/rllib/new_api_stack.rst

.. _multi-agent-episode-reference-docs:

MultiAgentEpisode API
=====================

.. include:: /_includes/rllib/new_api_stack.rst

rllib.env.multi_agent_episode.MultiAgentEpisode
-----------------------------------------------

Expand Down
4 changes: 2 additions & 2 deletions doc/source/rllib/package_ref/env/single_agent_env_runner.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,12 +1,12 @@
.. include:: /_includes/rllib/we_are_hiring.rst

.. include:: /_includes/rllib/new_api_stack.rst

.. _single-agent-env-runner-reference-docs:

SingleAgentEnvRunner API
========================

.. include:: /_includes/rllib/new_api_stack.rst

rllib.env.single_agent_env_runner.SingleAgentEnvRunner
------------------------------------------------------

Expand Down
4 changes: 2 additions & 2 deletions doc/source/rllib/package_ref/env/single_agent_episode.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,12 +1,12 @@
.. include:: /_includes/rllib/we_are_hiring.rst

.. include:: /_includes/rllib/new_api_stack.rst

.. _single-agent-episode-reference-docs:

SingleAgentEpisode API
======================

.. include:: /_includes/rllib/new_api_stack.rst

rllib.env.single_agent_episode.SingleAgentEpisode
-------------------------------------------------

Expand Down
4 changes: 2 additions & 2 deletions doc/source/rllib/package_ref/env/utils.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,12 +1,12 @@
.. include:: /_includes/rllib/we_are_hiring.rst

.. include:: /_includes/rllib/new_api_stack.rst

.. _env-utils-reference-docs:

Env Utils
=========

.. include:: /_includes/rllib/new_api_stack.rst

rllib.env.utils
---------------

Expand Down
5 changes: 2 additions & 3 deletions doc/source/rllib/package_ref/index.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,13 +1,12 @@

.. include:: /_includes/rllib/we_are_hiring.rst

.. include:: /_includes/rllib/new_api_stack.rst

.. _rllib-reference-docs:

Ray RLlib API
=============

.. include:: /_includes/rllib/new_api_stack.rst

.. tip:: We'd love to hear your feedback on using RLlib - `sign up to our forum and start asking questions <https://discuss.ray.io>`_!

This section contains an overview of RLlib's package- and API reference.
Expand Down
6 changes: 2 additions & 4 deletions doc/source/rllib/package_ref/learner.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,14 +1,12 @@

.. include:: /_includes/rllib/we_are_hiring.rst

.. include:: /_includes/rllib/new_api_stack.rst

.. _learner-reference-docs:


LearnerGroup API
================

.. include:: /_includes/rllib/new_api_stack.rst

Configuring a LearnerGroup and Learner actors
---------------------------------------------

Expand Down
6 changes: 2 additions & 4 deletions doc/source/rllib/package_ref/offline.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,14 +1,12 @@

.. include:: /_includes/rllib/we_are_hiring.rst

.. include:: /_includes/rllib/new_api_stack.rst

.. _new-api-offline-reference-docs:


Offline RL API
==============

.. include:: /_includes/rllib/new_api_stack.rst

Configuring Offline RL
----------------------

Expand Down
5 changes: 2 additions & 3 deletions doc/source/rllib/package_ref/replay-buffers.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,13 +1,12 @@

.. include:: /_includes/rllib/we_are_hiring.rst

.. include:: /_includes/rllib/new_api_stack.rst

.. _replay-buffer-api-reference-docs:

Replay Buffer API
=================

.. include:: /_includes/rllib/new_api_stack.rst

The following classes don't take into account the separation of experiences from different policies, multi-agent replay buffers will be explained further below.

Replay Buffer Base Classes
Expand Down
4 changes: 1 addition & 3 deletions doc/source/rllib/package_ref/rl_modules.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,13 +1,11 @@

.. include:: /_includes/rllib/we_are_hiring.rst

.. include:: /_includes/rllib/new_api_stack.rst

.. _rlmodule-reference-docs:

RLModule APIs
=============

.. include:: /_includes/rllib/new_api_stack.rst

RLModule specifications and configurations
-------------------------------------------
Expand Down
5 changes: 2 additions & 3 deletions doc/source/rllib/package_ref/utils.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,13 +1,12 @@

.. include:: /_includes/rllib/we_are_hiring.rst

.. include:: /_includes/rllib/new_api_stack.rst

.. _utils-reference-docs:

RLlib Utilities
===============

.. include:: /_includes/rllib/new_api_stack.rst

Here is a list of all the utilities available in RLlib.

Scheduler API
Expand Down
Loading
Loading