resolve v0.4 changes in doc

doc/source/basics/cli.rst

@@ -125,4 +125,4 @@ See More
 
 For more technical details about implementing custom CLIs and extensions, see:
 
-- :doc:`/reference/api/fairseq2.recipes/cli`
+- :doc:`/reference/api/fairseq2.cli/index`

doc/source/reference/api/fairseq2.recipes/cli.rst → doc/source/reference/api/fairseq2.cli/index.rst

@@ -1,8 +1,14 @@
-====================
-fairseq2.recipes.cli
-====================
+============
+fairseq2.cli
+============
+
+.. currentmodule:: fairseq2.cli
+
+.. toctree::
+    :maxdepth: 1
+
+    llama/index
 
-.. currentmodule:: fairseq2.recipes.cli
 
 Classes
 -------
@@ -19,9 +25,10 @@ Classes
 .. autoclass:: CliCommandHandler
    :members:
 
-.. autoclass:: RecipeCommandHandler
+.. autoclass:: fairseq2.cli.commands.recipe.RecipeCommandHandler
    :members:
 
+
 Examples
 --------
 
@@ -38,7 +45,8 @@ Here's a complete example:
 
 .. code-block:: python
 
-    from fairseq2.recipes.cli import Cli, CliCommandHandler, RecipeCommandHandler
+    from fairseq2.cli import Cli, CliCommandHandler
+    from fairseq2.cli.commands.recipe import RecipeCommandHandler
     
     def setup_custom_cli(cli: Cli) -> None:
         # Create a new command group
@@ -47,21 +55,31 @@ Here's a complete example:
             help="Custom recipes and utilities"
         )
         
-        # Add a command using RecipeCommandHandler
-        custom_handler = RecipeCommandHandler(
-            loader=load_custom_recipe,      # this is the recipe entrypoint callback function
-            preset_configs=custom_presets,  # this is the preset configs registry
-            default_preset="default",       # this is the default preset name
-            sweep_allowed_keys=["model", "dataset"]  # Optional
-        )
-        
         group.add_command(
             name="custom_command",
-            handler=custom_handler,
+            handler=custom_handler(),  # this is the command handler fn.
             help="Run custom recipe"
         )
 
-You can find more examples in our recipe examples:
+    def setup_recipe_cli(cli: Cli) -> None:
+        # Create a new command group for recipes
+        group = cli.add_group("recipe_name", help="Recipe commands")
+        
+        # create a recipe command handler first
+        recipe_handler = RecipeCommandHandler(
+            loader=recipe_loader,
+            config_kls=recipe_config_kls,
+            default_preset="recipe_default_preset",
+        )
+
+        group.add_command(
+            name="recipe_command_name",
+            handler=recipe_handler,
+            help="recipe_command_help",
+        )
+
+
+You can find more examples in our recipe command examples:
 
 * :mod:`fairseq2.recipes.lm.instruction_finetune`
 * :mod:`fairseq2.recipes.llama.convert_checkpoint`

doc/source/reference/api/fairseq2.recipes/llama/convert_checkpoint.rst → doc/source/reference/api/fairseq2.cli/llama/index.rst

@@ -1,12 +1,12 @@
-.. _reference-recipes-lm-convert-checkpoint:
+.. _reference-cli-llama:
 
 ===================
 Convert Checkpoints
 ===================
 
-.. module:: fairseq2.recipes.llama.convert_checkpoint
+.. module:: fairseq2.cli.commands.llama
 
-The checkpoint conversion module provides utilities to convert fairseq2 model checkpoints to different formats for interoperability with other frameworks.
+The checkpoint conversion handlers provides utilities to convert fairseq2 model checkpoints to different formats for interoperability with other frameworks.
 
 Command Line Interface
 ----------------------
@@ -74,23 +74,27 @@ Usage Example
 
 2. Convert to HuggingFace format:
 
-After converting to reference format, use the HuggingFace conversion script to convert to HF format:
-
 .. code-block:: bash
 
-    python -m transformers.models.llama.convert_llama_weights_to_hf \
-        --input_dir /path/to/output/dir \
-        --model_size 1B \
-        --output_dir /path/to/hf/model
+    fairseq2 llama write_hf_config --model <architecture> <fairseq2_checkpoint_dir>
+
+* ``<architecture>``: Specify the architecture of the model -- `e.g.`, ``llama3`` (see :mod:`fairseq2.models.llama`)
+
+* ``<fairseq2_checkpoint_dir>``: Path to the directory containing your Fairseq2 checkpoint, where ``config.json`` will be added.
+
+.. note::
+
+    Architecture ``--model`` must exist and be defined in `e.g.` :meth:`fairseq2.models.llama._config.register_llama_configs`.
 
 API Details
 -----------
 
-.. autoclass:: ConvertCheckpointCommandHandler
+.. autoclass:: ConvertLLaMACheckpointHandler
+
+.. autoclass:: WriteHFLLaMAConfigHandler
 
 See Also
 --------
 
 - :doc:`End-to-End Fine-Tuning Tutorial </tutorials/end_to_end_fine_tuning>`
-- :class:`fairseq2.models.llama.factory.LLaMAConfig`
-- :class:`fairseq2.models.llama.archs`
+- :class:`fairseq2.models.llama._config.LLaMAConfig`

doc/source/reference/api/fairseq2.recipes/index.rst

@@ -11,7 +11,5 @@ different model architectures.
     :maxdepth: 1
 
     lm/index
-    llama/index
     wav2vec2/index
-    cli
     trainer

doc/source/reference/api/fairseq2.recipes/lm/index.rst

@@ -16,17 +16,105 @@ Key Features
 - CLI setup for language model training
 - Common training recipes and configurations
 
-Submodules
-==========
+Instruction Fine-Tuning
+=======================
 
-.. toctree::
-    :maxdepth: 1
+Classes
+~~~~~~~
 
-    instruction_finetune
+.. autoclass:: InstructionFinetuneConfig
+
+.. autoclass:: InstructionFinetuneCriterion
+
+.. autoclass:: InstructionFinetuneDatasetSection
+
+.. autoclass:: InstructionFinetuneUnit
+
+.. autoclass:: InstructionLossEvalUnit
+
+
+Functions
+~~~~~~~~~
+
+.. autofunction:: load_instruction_finetuner
+
+
+Preference Fine-Tuning
+======================
+
+Classes (PO)
+~~~~~~~~~~~~
+
+.. autoclass:: POFinetuneConfig
+
+.. autoclass:: POFinetuneMetricBag
+
+.. autoclass:: POCriterionSection
+
+.. autoclass:: POFinetuneDatasetSection
+
+
+.. autoclass:: CpoFinetuneConfig
+
+.. autoclass:: CpoFinetuneUnit
+
+.. autoclass:: CpoFinetuneUnitHandler
+
+.. autoclass:: CpoFinetuneMetricBag
+
+
+.. autoclass:: DpoFinetuneConfig
+
+.. autoclass:: DpoFinetuneUnit
+
+.. autoclass:: DpoFinetuneUnitHandler
+
+.. autoclass:: DpoFinetuneMetricBag
+
+
+.. autoclass:: OrpoFinetuneConfig
+
+.. autoclass:: OrpoFinetuneUnit
+
+.. autoclass:: OrpoFinetuneUnitHandler
+
+.. autoclass:: OrpoFinetuneMetricBag
+
+
+.. autoclass:: SimPOFinetuneConfig
+
+.. autoclass:: SimPOFinetuneUnit
+
+.. autoclass:: SimPOFinetuneUnitHandler
+
+.. autoclass:: SimPOFinetuneMetricBag
+
+Functions (PO)
+~~~~~~~~~~~~~~
+
+.. autofunction:: load_po_finetuner
+
+
+Text Generation
+================
+
+Classes (Text Generation)
+~~~~~~~~~~~~~~~~~~~~~~~~~
+
+.. autoclass:: TextGenerateConfig
+
+.. autoclass:: TextGenerateDatasetSection
+
+.. autoclass:: TextGenerateUnit
+
+Functions (Text Generation)
+~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+.. autofunction:: load_text_generator
 
-The ``instruction_finetune`` module provides specialized utilities for instruction-based fine-tuning of language models.
 
 Usage Examples
 ==============
 
 - :ref:`tutorial-end-to-end-fine-tuning`
+

doc/source/reference/api/fairseq2.recipes/wav2vec2/index.rst

@@ -4,14 +4,27 @@ fairseq2.recipes.wav2vec2
 
 .. module:: fairseq2.recipes.wav2vec2
 
+.. autoclasstree:: fairseq2.recipes.wav2vec2
+   :full:
+   :zoom:
 
-Functions
-=========
+Classes
+=======
+
+.. autoclass:: Wav2Vec2TrainConfig
+
+.. autoclass:: Wav2Vec2TrainUnit
+
+.. autoclass:: Wav2Vec2EvalConfig
 
-.. autofunction:: _setup_wav2vec2_cli
+.. autoclass:: Wav2Vec2MetricBag
 
+.. autoclass:: Wav2Vec2Criterion
+
+
+Functions
+=========
 
-.. toctree::
-    :maxdepth: 1
+.. autofunction:: load_wav2vec2_trainer
 
-    train
+.. autofunction:: load_wav2vec2_evaluator

doc/source/reference/api/index.rst

@@ -8,6 +8,7 @@
 
     fairseq2
     gang
+    fairseq2.cli/index
     fairseq2.data/index
     fairseq2.datasets/index
     fairseq2.nn/index