Skip to content
This repository has been archived by the owner on Oct 25, 2024. It is now read-only.

Commit

Permalink
Update openai-python api usage (#1275)
Browse files Browse the repository at this point in the history
Signed-off-by: lvliang-intel <liang1.lv@intel.com>
  • Loading branch information
lvliang-intel authored Feb 15, 2024
1 parent f565baa commit 494910c
Show file tree
Hide file tree
Showing 2 changed files with 19 additions and 10 deletions.
21 changes: 14 additions & 7 deletions intel_extension_for_transformers/neural_chat/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -74,20 +74,27 @@ server_executor(config_file="./server/config/neuralchat.yaml", log_file="./neura
Once the service is running, you can observe an OpenAI-compatible endpoint `/v1/chat/completions`. You can use any of below ways to access the endpoint.

#### Using OpenAI Client Library

First, install openai-python:

```bash
pip install --upgrade openai
```

Then, interact with the model:

```python
from openai import Client
# Replace 'your_api_key' with your actual OpenAI API key
api_key = 'your_api_key'
backend_url = 'http://127.0.0.1:80/v1/chat/completions'
client = Client(api_key=api_key, base_url=backend_url)
response = client.ChatCompletion.create(
import openai
openai.api_key = "EMPTY"
openai.base_url = 'http://127.0.0.1:80/v1/'
response = openai.chat.completions.create(
model="Intel/neural-chat-7b-v3-1",
messages=[
{"role": "system", "content": "You are a helpful assistant."},
{"role": "user", "content": "Tell me about Intel Xeon Scalable Processors."},
]
)
print(response)
print(response.choices[0].message.content)
```

#### Using Curl
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -52,23 +52,25 @@ First, install openai-python:
pip install --upgrade openai
```

Then, interact with the model:

```python
import openai
# to get proper authentication, make sure to use a valid key that's listed in
# the --api-keys flag. if no flag value is provided, the `api_key` will be ignored.
openai.api_key = "EMPTY"
openai.api_base = "http://localhost:80/v1"
openai.base_url = "http://localhost:80/v1/"

model = "Intel/neural-chat-7b-v3-1"
prompt = "Once upon a time"

# create a completion
completion = openai.Completion.create(model=model, prompt=prompt, max_tokens=64)
completion = openai.completions.create(model=model, prompt=prompt, max_tokens=64)
# print the completion
print(prompt + completion.choices[0].text)

# create a chat completion
completion = openai.ChatCompletion.create(
completion = openai.chat.completions.create(
model=model,
messages=[{"role": "user", "content": "Tell me about Intel Xeon Scalable Processors."}]
)
Expand Down

0 comments on commit 494910c

Please sign in to comment.