From 6cc6a2278e99ab14468773a8ca41bc98041887f4 Mon Sep 17 00:00:00 2001 From: Liangyx2 Date: Wed, 22 Nov 2023 16:47:37 +0800 Subject: [PATCH] update build_chatbot_on_xpu.ipynb Signed-off-by: Liangyx2 --- .../docs/notebooks/build_chatbot_on_xpu.ipynb | 275 ++++++++++++++++++ 1 file changed, 275 insertions(+) create mode 100644 intel_extension_for_transformers/neural_chat/docs/notebooks/build_chatbot_on_xpu.ipynb diff --git a/intel_extension_for_transformers/neural_chat/docs/notebooks/build_chatbot_on_xpu.ipynb b/intel_extension_for_transformers/neural_chat/docs/notebooks/build_chatbot_on_xpu.ipynb new file mode 100644 index 00000000000..ada9c0e8d7a --- /dev/null +++ b/intel_extension_for_transformers/neural_chat/docs/notebooks/build_chatbot_on_xpu.ipynb @@ -0,0 +1,275 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "NeuralChat is a customizable chat framework designed to create user own chatbot within few minutes on multiple architectures. This notebook is used to demostrate how to build a talking chatbot on Intel® Data Center GPU Flex Series 170, Intel® Data Center GPU Max Series and Intel® Arc™ A-Series GPUs." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Prepare Environment" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Install requirements" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "!git clone https://github.com/intel/intel-extension-for-transformers.git" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "%cd ./intel-extension-for-transformers/\n", + "!pip install -r requirements.txt\n", + "%cd ./intel_extension_for_transformers/neural_chat/\n", + "!pip install -r requirements_xpu.txt" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Install Intel® Extension for Transformers*" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "%cd ../../\n", + "!pip install v ." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Refer to [Download oneapi](https://www.intel.com/content/www/us/en/developer/tools/oneapi/base-toolkit-download.html) to install oneapi base toolkit, and run the command below." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "!source /opt/intel/oneapi/setvars.sh" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Refer to [Install Intel® Extension for PyTorch* from source](https://intel.github.io/intel-extension-for-pytorch/index.html#installation) to build xpu version of torch, torchaudio and Intel® Extension for PyTorch*, and install generated wheels using pip." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Install requirements that have denpendency on stock pytorch" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "!pip install --no-deps peft speechbrain optimum optimum-intel sentence_transformers lm_eval accelerate" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Notes: If you face \"GLIBCXX_3.4.30\" not found issue in conda environment, please remove lib/libstdc++* from conda environment. " + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Prepare the model" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Make sure to request access at https://huggingface.co/meta-llama/Llama-2-7b-chat-hf and pass a token having permission to this repo either by logging in with huggingface-cli login or by passing token=." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Inference 💻" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Text Chat" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Giving NeuralChat the textual instruction, it will respond with the textual response." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "from intel_extension_for_transformers.neural_chat import build_chatbot\n", + "from intel_extension_for_transformers.neural_chat import PipelineConfig\n", + "config = PipelineConfig(device='xpu')\n", + "chatbot = build_chatbot(config)\n", + "response = chatbot.predict(\"Tell me about Intel Xeon Scalable Processors.\")\n", + "print(response)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Text Chat With RAG Plugin" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "User could also leverage NeuralChat RAG plugin to do domain specific chat by feding with some documents like below" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "!mkdir docs\n", + "%cd docs\n", + "!curl -OL https://raw.githubusercontent.com/intel/intel-extension-for-transformers/main/intel_extension_for_transformers/neural_chat/assets/docs/4th Generation Intel® Xeon® Scalable Processors Product Specifications.html\n", + "!curl -OL https://raw.githubusercontent.com/intel/intel-extension-for-transformers/main/intel_extension_for_transformers/neural_chat/assets/docs/sample.jsonl\n", + "!curl -OL https://raw.githubusercontent.com/intel/intel-extension-for-transformers/main/intel_extension_for_transformers/neural_chat/assets/docs/sample.txt\n", + "!curl -OL https://raw.githubusercontent.com/intel/intel-extension-for-transformers/main/intel_extension_for_transformers/neural_chat/assets/docs/sample.xlsx\n", + "%cd .." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "from intel_extension_for_transformers.neural_chat import PipelineConfig\n", + "from intel_extension_for_transformers.neural_chat import build_chatbot\n", + "from intel_extension_for_transformers.neural_chat import plugins\n", + "plugins.retrieval.enable=True\n", + "plugins.retrieval.args[\"input_path\"]=\"./docs/\"\n", + "config = PipelineConfig(plugins=plugins, device='xpu')\n", + "chatbot = build_chatbot(config)\n", + "response = chatbot.predict(\"How many cores does the Intel® Xeon® Platinum 8480+ Processor have in total?\")\n", + "print(response)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Voice Chat with ATS & TTS Plugin" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "In the context of voice chat, users have the option to engage in various modes: utilizing input audio and receiving output audio, employing input audio and receiving textual output, or providing input in textual form and receiving audio output.\n", + "\n", + "For the Python API code, users have the option to enable different voice chat modes by setting ASR and TTS plugins enable or disable." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "!curl -OL https://raw.githubusercontent.com/intel/intel-extension-for-transformers/main/intel_extension_for_transformers/neural_chat/assets/speaker_embeddings/spk_embed_default.pt\n", + "!curl -OL https://raw.githubusercontent.com/intel/intel-extension-for-transformers/main/intel_extension_for_transformers/neural_chat/assets/audio/sample.wav" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "from intel_extension_for_transformers.neural_chat import PipelineConfig\n", + "from intel_extension_for_transformers.neural_chat import build_chatbot, plugins\n", + "plugins.asr.enable = True\n", + "plugins.tts.enable = True\n", + "plugins.tts.args[\"output_audio_path\"]=\"./output_audio.wav\"\n", + "config = PipelineConfig(plugins=plugins, device='xpu')\n", + "chatbot = build_chatbot(config)\n", + "result = chatbot.predict(query=\"./sample.wav\")\n", + "print(result)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "You can display the generated wav file using IPython." + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.10.0" + } + }, + "nbformat": 4, + "nbformat_minor": 4 +}