Distillation qanda (#3335)

* Q and A Distillation Example * Create distillation_qanda - Copy.ipynb Distillation Q and A changes * Changes for distillation Q&A * Changes for black validation and utils for download dataset
Azure · Aug 13, 2024 · a46bb8e · a46bb8e
1 parent ce99aa8
commit a46bb8e
Show file tree

Hide file tree

Showing 3 changed files with 660 additions and 0 deletions.
diff --git a/...on-models/system/finetune/Llama-notebooks/distillation/distillation_chat_completion.ipynb b/...on-models/system/finetune/Llama-notebooks/distillation/distillation_chat_completion.ipynb
@@ -486,6 +486,7 @@
     "        learning_rate=0.00002,\n",
     "        per_device_train_batch_size=1,\n",
     "        num_train_epochs=3,\n",
+    "        data_generation_task_type=\"NLI\",\n",
     "    )\n",
     "\n",
     "    return {\"output_model\": oss_distillation.outputs.output_model}"