port docs folder to azureml-examples from foundational repo (#2440)

* port docs folder to azureml-examples from foundational repo

* repair relative path

---------

Co-authored-by: grajguru <grajguru@microsoft.com>

sdk/python/foundation-models/system/docs/component_docs/evaluate_model.md

@@ -0,0 +1,131 @@
+# LLM Evaluate Model Component
+This component enables user to evaluate a model by providing the supported model, run inference to generate predictions first followed by computing metrics against a dataset. You can find the component in your workspace components page. ![as shown in the figure](../images/evaluate_model_component.png)
+
+# Inputs
+1. _task_ (string, required):
+
+    Task type for which model is trained
+2. _test_data_ (URI_FILE, required):
+
+    Path to file containing test data in `jsonl` format
+3. _evaluation_config_ (URI_FILE, optional):
+
+    Path to file containing test data in `jsonl` format
+4. _test_data_input_column_names_ (string, required):
+
+    Name of the columns in the test dataset that should be used for prediction. More than one columns should be separated by the comma(,) delimiter without any whitespaces in between
+4. _test_data_label_column_name_ (string, required):
+
+    Name of the key containing target values in test data.
+5. _mlflow_model_ (mlflow_model, optional):
+
+    MLFlow model (either registered or output of another job)
+6. _model_uri_ (string, optional):
+
+    MLFlow model uri of the form - <br>
+    fetched from azureml run as `runs:/<azureml_run_id>/run-relative/path/to/model` <br>
+    fetched from azureml model registry as `models:/<model_name>/<model_version>`
+7. _device_ (string, optional):
+
+    Option to run the experiment on CPU or GPU provided that the compute that they are choosing has Cuda support.
+8. _batch_size_ (integer, optional):
+
+    Option to provide batch size for a task.
+
+
+# Outputs
+1. _evaluation_result_ (URI_FOLDER):
+
+    Path to output directory which contains the generated predictions.csv file containing predictions for the provided test set to the component and other metadata in the evaluationResult folder
+
+# Additional Parameters
+- Classification Inputs
+    1. _metrics_ (list[str], optional): 
+
+        List of metric names to be computed. If not provided we choose the default set. 
+    2. _class_labels_ (list[any], optional): 
+
+        List of labels for entire data (all the data train/test/validation) 
+    3. _train_labels_ (list[any], optional): 
+
+        List of labels used during training. 
+    4. _sample_weight_ (list[float], optional):  
+        Weights for the samples (Does not need to match sample weights on the fitted model) 
+    6. _use_binary_ (bool, optional): 
+
+        Boolean argument on whether to use binary classification metrics or not 
+    7. _positive_label_ (any, optional):
+
+        Class designed as positive class in binary classification metrics. 
+    8. _multilabel_ (bool, optional): 
+
+        Whether the classification type is multilabel or single label.
+
+- Regression Inputs
+    1. _metrics_ (list[str], optional): 
+
+        List of metric names to be computed. If not provided we choose the default set. 
+    2. _y_max_ (float, optional): 
+
+        The max target value. 
+    3. _y_min_ (float, optional): 
+
+        The min target value. 
+    4. _y_std_ (float, optional): 
+
+        The standard deviation of targets value. 
+    5. _sample_weight_ (list[float], optional):
+
+        Weights for the samples (Does not need to match sample weights on the fitted model) 
+    6. _bin_info_ (dict[str, float], optional):  
+
+        The binning information for true values. This should be calculated from make_dataset_bins. Required for 
+        calculating non-scalar metrics. 
+
+- Token Classification Inputs
+    1. _train_label_list_ (list[str], optional)
+
+        List of labels for training data.
+    2. _metrics_ (list[str], optional)
+
+        List of metric names to be computed. If not provided we choose the default set. 
+
+- Summarization Inputs
+    1. _metrics_ (list[str], optional)
+
+        List of metric names to be computed. If not provided we choose the default set. 
+    2. _aggregator_ (bool, optional)
+
+        Boolean to indicate whether to aggregate scores.
+    3. _stemmer_ (bool, optional)
+
+         Boolean to indicate whether to use Porter stemmer for word suffixes.
+
+- Translation Inputs
+    1. _metrics_ (list[str], optional)
+
+        List of metric names to be computed. If not provided we choose the default set. 
+    2. _smoothing_ (bool, optional)
+
+        Boolean to indicate whether to smooth out the bleu score
+
+- Question Answering Inputs
+    1. _metrics_ (list[str], optional)
+
+        List of metric names to be computed. If not provided we choose the default set. 
+    2. _regexes_to_ignore_ (list[str], optional)
+
+        List of string regular expressions to ignore.
+    3. _ignore_case_ (bool, optional)
+
+         Boolean to indicate whether to ignore case.
+    4. _ignore_punctuation_ (bool, optional)
+
+         Boolean to indicate whether to ignore punctuation.
+    5. _ignore_numbers_ (bool, optional)
+
+         Boolean to indicate whether to ignore numbers.
+
+
+# Caveats, Recommendations and Known Issues
+Some AutoML Models are not generating correct Mlflow-models so they might not work on with Evaluate Model Component. 

sdk/python/foundation-models/system/docs/component_docs/finetune_component.md

@@ -0,0 +1,204 @@
+# Finetune Component
+This component enables finetuning of pretrained models on custom or pre-available datasets. The component supports LoRA, Deepspeed and ONNXRuntime configurations for performance enhancement. 
+The components can be seen here ![as shown in the figure](../images/finetuning_components.png)
+
+# 1. Inputs
+1. _deepspeed_config_ (URI_FILE, optional)
+
+    Path to the deepspeed config file
+
+2. _dataset_path_ (URI_FOLDER, required)
+
+    Path to the output folder of [preprocess component](preprocess_component.md/#2-outputs)
+
+3. _model_path_ (URI_FOLDER, required)
+
+    Path to the output directory of [model_selector component](model_selector_component.md/#2-outputs)
+
+
+# 2. Outputs
+1. _output_dir_ (URI_FOLDER)
+
+    The folder containing finetuned model output with checkpoints, model configs, tokenizers, optimzer and scheduler states and random number states in case of distributed training.
+
+
+# 3. Parameters
+
+## 3.1. Lora Parameters
+1. _apply_lora_ (string, optional)
+
+    If "true" enables lora. The default is "false". The lora is `ONLY` supported for models present in `model_id` dropdown of `model_selector` component. When *finetune component* is connected to *model_evaluation* component, _merge_lora_weights_ **MUST** be set to "true" when _apply_lora_ is "true"
+
+2. _merge_lora_weights_ (string, optional)
+
+    If "true", the lora weights are merged with the base Hugging Face model. The default value is "false"
+
+2. _lora_alpha_ (integer, optional)
+
+    alpha attention parameter for lora
+
+3. _lora_r_ (integer, optional)
+
+    The rank to be used with lora.
+
+4. _lora_dropout_ (float, optional)
+
+    lora dropout value
+
+## 3.2. Training Parameters
+
+1. _epochs_ (int, optional)
+
+    Number of epochs to run for finetune. The default value is 5
+
+2. _max_steps_ (int, optional)
+
+    If set to a positive number, the total number of training steps to perform. Overrides `epochs`. In case of using a finite iterable dataset the training may stop before reaching the set number of steps when all data is exhausted. default value is -1
+
+3. _learning_rate_ (float, optional)
+
+    Start learning rate used for training. default value is 2e-5
+
+4. _train_batch_size_ (int, optional)
+
+    Batch size used for training. default value is 4.
+
+5. _valid_batch_size_ (int, optional)
+
+    Batch size used for validation. default value is 4.
+
+6. _auto_find_batch_size_ (string, optional)
+
+    If set to "true", the train batch size will be automatically downscaled recursively till if finds a valid batch size that fits into memory. The default value is "false".
+
+## 3.3. Deepspeed and ORT Parameters
+1. _apply_ort_ (string, optional)
+
+    If "true" apply ORT optimization. The default is "false".
+
+2. _apply_deepspeed_ (string, optional)
+
+    If "true" enables deepspeed. If no `deepspeed_config` is provided, the default config in the component will be used else the user passed config will be used. The default is "false".
+
+    Please note that to enable deepspeed, `apply_deepspeed` must be set to true
+
+
+## 3.4. Optimizer and Scheduler Parameters
+
+1. _optimizer_ (string, optional)
+
+    Optimizer to be used while training. default value is "adamw_hf"
+
+2. _warmup_steps_ (int, optional)
+
+    The number of steps for the learning rate scheduler warmup phase. The default value is 20
+
+3. _weight_decay_ (float, optional)
+
+    The weight decay to apply (if not zero) to all layers except all bias and LayerNorm weights in AdamW optimizer. The default value is 0
+
+4. _adam_beta1_ (float, optional)
+
+    The beta1 hyperparameter for the AdamW optimizer. The default value is 0.9
+
+5. _adam_beta2_ (float, optional)
+
+    The beta2 hyperparameter for the AdamW optimizer. The default value is 0.999
+
+6. _adam_epsilon_ (float, optional)
+
+    The epsilon hyperparameter for the AdamW optimizer. The default value is 1e-8
+
+7. _gradient_accumulation_steps_ (int, optional)
+
+    Number of updates steps to accumulate the gradients for, before performing a backward/update pass. The default value is 1
+
+8. _lr_scheduler_type_ (string, optional)
+
+    The learning rate scheduler to use. The default value is `linear`
+
+## 3.5. Misc Parameters
+
+1. _precision_ (int, optional)
+
+    Apply mixed precision training. This can reduce memory footprint by performing operations in half-precision. The default value is 32
+
+2. _seed_ (int, optional)
+
+    Random seed that will be set at the beginning of training. The default value is 42
+
+3. _evaluation_strategy_ (string, optional)
+
+    The evaluation strategy to adopt during training. If set to "steps", either the `evaluation_steps_interval` or `evaluation_steps` needs to be specified, which helps to determine the step at which the model evaluation needs to be computed else evaluation happens at end of each epoch. The default value is "epoch"
+
+4. _evaluation_steps_interval_ (float, optional)
+
+    The evaluation steps in fraction of an epoch steps to adopt during training. Overwrites evaluation_steps if not 0. The default value is 0
+
+5. _evaluation_steps_ (int, optional)
+
+    Number of update steps between two model evaluations if evaluation_strategy='steps'. The default value is 500
+
+6. _logging_strategy_ (string, optional)
+
+    The logging strategy to adopt during training. If set to "steps", the `logging_steps` will decide the frequency of logging else logging happens at the end of epoch. The default value is "epoch".
+
+7. _logging_steps_ (int, optional)
+
+    Number of update steps between two logs if logging_strategy='steps'. The default value is 100
+
+8. _save_total_limit_ (int, optional)
+
+    If a value is passed, will limit the total amount of checkpoints. Deletes the older checkpoints in output_dir. If the value is -1 saves all checkpoints". The default value is -1
+
+9. _apply_early_stopping_ (string, optional)
+
+    If set to "true", early stopping is enabled. The default value is "false"
+
+10. _early_stopping_patience_ (int, optional)
+
+    Stop training when the specified metric worsens for early_stopping_patience evaluation calls.
+
+11. _early_stopping_threshold_ (float, optional)
+
+    Denotes how much the specified metric must improve to satisfy early stopping conditions.
+
+## 3.6. Continual Finetuning
+
+1. _resume_from_checkpoint_ (string, optional)
+
+    If set to "true", resumes the training from last saved checkpoint. Along with loading the saved weights, saved optimizer, scheduler and random states will be loaded if exists. The default value is "false"
+
+
+# 4. Run Settings
+
+This setting helps to choose the compute for running the component code. **For the purpose of finetune, gpu compute should be used**. We recommend using ND24 compute.
+
+> Select *Use other compute target*
+
+- Under this option, you can select either `compute_cluster` or `compute_instance` as the compute type and the corresponding instance / cluster created in your workspace.
+- If you have not created the compute, you can create the compute by clicking the `Create Azure ML compute cluster` link that's available while selecting the compute. See the figure below
+![other compute target](../images/other_compute_target.png)
+
+
+## 4.1. Settings for Distributed Training
+
+> When creating the compute, set the `Maximum number of nodes` to the desired value for multi-node training as shown in the figure below
+
+![set maximum nodes](../images/maximum_num_nodes.png)
+
+> In case of distributed training, a.k.a multi-node training, the mode must be set to `Mount` and not `Upload` as shown in the figure below
+
+![Output settings finetune](../images/output_settings_mount.png)
+
+> Set the number of processes under Distribution subsection to use all the gpus in a node
+
+To use all the gpus within a node, set the `Process count per instance` to number of gpus in that node as shown below
+
+![process count per instance](../images/process_count_per_instance.png)
+
+> Set the number of nodes under the Resources subsection
+
+In case of distributed training, you can configure `instance count` under this subsection to increase the number of nodes as shown below
+
+![instance count](../images/instance_count.png)