Skip to content
This repository has been archived by the owner on Oct 25, 2024. It is now read-only.

Commit

Permalink
update build_chatbot_on_xpu.ipynb
Browse files Browse the repository at this point in the history 
Signed-off-by: Liangyx2 <yuxiang.liang@intel.com>
  • Loading branch information
@Liangyx2
Liangyx2 authored Nov 22, 2023
1 parent 96dc825 commit 6cc6a22
Showing 1 changed file with 275 additions and 0 deletions.
275 changes: 275 additions & 0 deletions intel_extension_for_transformers/neural_chat/docs/notebooks/build_chatbot_on_xpu.ipynb
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,275 @@
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"NeuralChat is a customizable chat framework designed to create user own chatbot within few minutes on multiple architectures. This notebook is used to demostrate how to build a talking chatbot on Intel® Data Center GPU Flex Series 170, Intel® Data Center GPU Max Series and Intel® Arc™ A-Series GPUs."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Prepare Environment"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Install requirements"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"!git clone https://github.com/intel/intel-extension-for-transformers.git"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"%cd ./intel-extension-for-transformers/\n",
"!pip install -r requirements.txt\n",
"%cd ./intel_extension_for_transformers/neural_chat/\n",
"!pip install -r requirements_xpu.txt"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Install Intel® Extension for Transformers*"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"%cd ../../\n",
"!pip install v ."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Refer to [Download oneapi](https://www.intel.com/content/www/us/en/developer/tools/oneapi/base-toolkit-download.html) to install oneapi base toolkit, and run the command below."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"!source /opt/intel/oneapi/setvars.sh"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Refer to [Install Intel® Extension for PyTorch* from source](https://intel.github.io/intel-extension-for-pytorch/index.html#installation) to build xpu version of torch, torchaudio and Intel® Extension for PyTorch*, and install generated wheels using pip."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Install requirements that have denpendency on stock pytorch"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"!pip install --no-deps peft speechbrain optimum optimum-intel sentence_transformers lm_eval accelerate"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Notes: If you face \"GLIBCXX_3.4.30\" not found issue in conda environment, please remove lib/libstdc++* from conda environment. "
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Prepare the model"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Make sure to request access at https://huggingface.co/meta-llama/Llama-2-7b-chat-hf and pass a token having permission to this repo either by logging in with huggingface-cli login or by passing token=<your_token>."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Inference 💻"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Text Chat"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Giving NeuralChat the textual instruction, it will respond with the textual response."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"from intel_extension_for_transformers.neural_chat import build_chatbot\n",
"from intel_extension_for_transformers.neural_chat import PipelineConfig\n",
"config = PipelineConfig(device='xpu')\n",
"chatbot = build_chatbot(config)\n",
"response = chatbot.predict(\"Tell me about Intel Xeon Scalable Processors.\")\n",
"print(response)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Text Chat With RAG Plugin"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"User could also leverage NeuralChat RAG plugin to do domain specific chat by feding with some documents like below"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"!mkdir docs\n",
"%cd docs\n",
"!curl -OL https://raw.githubusercontent.com/intel/intel-extension-for-transformers/main/intel_extension_for_transformers/neural_chat/assets/docs/4th Generation Intel® Xeon® Scalable Processors Product Specifications.html\n",
"!curl -OL https://raw.githubusercontent.com/intel/intel-extension-for-transformers/main/intel_extension_for_transformers/neural_chat/assets/docs/sample.jsonl\n",
"!curl -OL https://raw.githubusercontent.com/intel/intel-extension-for-transformers/main/intel_extension_for_transformers/neural_chat/assets/docs/sample.txt\n",
"!curl -OL https://raw.githubusercontent.com/intel/intel-extension-for-transformers/main/intel_extension_for_transformers/neural_chat/assets/docs/sample.xlsx\n",
"%cd .."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"from intel_extension_for_transformers.neural_chat import PipelineConfig\n",
"from intel_extension_for_transformers.neural_chat import build_chatbot\n",
"from intel_extension_for_transformers.neural_chat import plugins\n",
"plugins.retrieval.enable=True\n",
"plugins.retrieval.args[\"input_path\"]=\"./docs/\"\n",
"config = PipelineConfig(plugins=plugins, device='xpu')\n",
"chatbot = build_chatbot(config)\n",
"response = chatbot.predict(\"How many cores does the Intel® Xeon® Platinum 8480+ Processor have in total?\")\n",
"print(response)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Voice Chat with ATS & TTS Plugin"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"In the context of voice chat, users have the option to engage in various modes: utilizing input audio and receiving output audio, employing input audio and receiving textual output, or providing input in textual form and receiving audio output.\n",
"\n",
"For the Python API code, users have the option to enable different voice chat modes by setting ASR and TTS plugins enable or disable."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"!curl -OL https://raw.githubusercontent.com/intel/intel-extension-for-transformers/main/intel_extension_for_transformers/neural_chat/assets/speaker_embeddings/spk_embed_default.pt\n",
"!curl -OL https://raw.githubusercontent.com/intel/intel-extension-for-transformers/main/intel_extension_for_transformers/neural_chat/assets/audio/sample.wav"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"from intel_extension_for_transformers.neural_chat import PipelineConfig\n",
"from intel_extension_for_transformers.neural_chat import build_chatbot, plugins\n",
"plugins.asr.enable = True\n",
"plugins.tts.enable = True\n",
"plugins.tts.args[\"output_audio_path\"]=\"./output_audio.wav\"\n",
"config = PipelineConfig(plugins=plugins, device='xpu')\n",
"chatbot = build_chatbot(config)\n",
"result = chatbot.predict(query=\"./sample.wav\")\n",
"print(result)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"You can display the generated wav file using IPython."
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.10.0"
}
},
"nbformat": 4,
"nbformat_minor": 4
}

0 comments on commit 6cc6a22

Please sign in to comment.