Skip to content
This repository has been archived by the owner on Oct 25, 2024. It is now read-only.

[NeuralChat] Update openai-python API usage for library API upgrade #1275

Merged
merged 1 commit into from
Feb 15, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 14 additions & 7 deletions intel_extension_for_transformers/neural_chat/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -74,20 +74,27 @@ server_executor(config_file="./server/config/neuralchat.yaml", log_file="./neura
Once the service is running, you can observe an OpenAI-compatible endpoint `/v1/chat/completions`. You can use any of below ways to access the endpoint.

#### Using OpenAI Client Library

First, install openai-python:

```bash
pip install --upgrade openai
```

Then, interact with the model:

```python
from openai import Client
# Replace 'your_api_key' with your actual OpenAI API key
api_key = 'your_api_key'
backend_url = 'http://127.0.0.1:80/v1/chat/completions'
client = Client(api_key=api_key, base_url=backend_url)
response = client.ChatCompletion.create(
import openai
openai.api_key = "EMPTY"
openai.base_url = 'http://127.0.0.1:80/v1/'
response = openai.chat.completions.create(
model="Intel/neural-chat-7b-v3-1",
messages=[
{"role": "system", "content": "You are a helpful assistant."},
{"role": "user", "content": "Tell me about Intel Xeon Scalable Processors."},
]
)
print(response)
print(response.choices[0].message.content)
```

#### Using Curl
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -52,23 +52,25 @@ First, install openai-python:
pip install --upgrade openai
```

Then, interact with the model:

```python
import openai
# to get proper authentication, make sure to use a valid key that's listed in
# the --api-keys flag. if no flag value is provided, the `api_key` will be ignored.
openai.api_key = "EMPTY"
openai.api_base = "http://localhost:80/v1"
openai.base_url = "http://localhost:80/v1/"

model = "Intel/neural-chat-7b-v3-1"
prompt = "Once upon a time"

# create a completion
completion = openai.Completion.create(model=model, prompt=prompt, max_tokens=64)
completion = openai.completions.create(model=model, prompt=prompt, max_tokens=64)
# print the completion
print(prompt + completion.choices[0].text)

# create a chat completion
completion = openai.ChatCompletion.create(
completion = openai.chat.completions.create(
model=model,
messages=[{"role": "user", "content": "Tell me about Intel Xeon Scalable Processors."}]
)
Expand Down
Loading