Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add o3-mini support #374

Closed
Nicba1010 opened this issue Jan 31, 2025 · 11 comments
Closed

Add o3-mini support #374

Nicba1010 opened this issue Jan 31, 2025 · 11 comments

Comments

@Nicba1010
Copy link

o3-mini support is missing
@tarekgh
Copy link

tarekgh commented Jan 31, 2025 •
edited
Loading

Is it known which vocab file the tokenizer is going to use with this model?

@Peter-Juhasz Peter-Juhasz mentioned this issue Feb 2, 2025
Open
@kashankrm
Copy link

Based on what i know it is the same tokenizer as gpt 4o. they just added some extra stuff to it.
I did a small test like this:

from openai import OpenAI

client = OpenAI() 

def generate_response(prompt: str, model: str = "o3-mini") -> str:
    print(f"Generating response for {model} model...")
    response = client.chat.completions.create(
        model=model,
        messages=[
        {"role": "user", "content": prompt}
    ],
    )
    # print(response.choices[0].message.content)
    print("input tokens:", response.usage.prompt_tokens)
    print("reasoning tokens:", response.usage.completion_tokens_details.reasoning_tokens)
    print("output tokens:", response.usage.completion_tokens - response.usage.completion_tokens_details.reasoning_tokens)
    print("\n ---------------- \n")
    return response.choices[0].message.content


messages = [
    "Complete this sentence: The quick brown fox jumps",
    "kajsh  ekr jn as kemnralsekjr la sekjrl skejlsmcelkamc .skenrs",
    "code, make sure you have installed the OpenAI Python"
]
for message in messages:
    print("testing for message:", message)
    generate_response(message)
    generate_response(message, model="o1")
    generate_response(message, model="o1-mini")
    generate_response(message, model="o1-preview")
    generate_response(message, model="gpt-4o")

here is the output:

testing for message: Complete this sentence: The quick brown fox jumps
Generating response for o3-mini model...
input tokens: 15
reasoning tokens: 192
output tokens: 20

 ---------------- 

Generating response for o1 model...
input tokens: 15
reasoning tokens: 256
output tokens: 21

 ---------------- 

Generating response for o1-mini model...
input tokens: 17
reasoning tokens: 128
output tokens: 22

 ---------------- 

Generating response for o1-preview model...
input tokens: 17
reasoning tokens: 128
output tokens: 29

 ---------------- 

Generating response for gpt-4o model...
input tokens: 16
reasoning tokens: 0
output tokens: 6

 ---------------- 

testing for message: kajsh  ekr jn as kemnralsekjr la sekjrl skejlsmcelkamc .skenrs
Generating response for o3-mini model...
input tokens: 34
reasoning tokens: 128
output tokens: 42

 ---------------- 

Generating response for o1 model...
input tokens: 34
reasoning tokens: 384
output tokens: 56

 ---------------- 

Generating response for o1-mini model...
input tokens: 36
reasoning tokens: 192
output tokens: 41

 ---------------- 

Generating response for o1-preview model...
input tokens: 36
reasoning tokens: 320
output tokens: 39

 ---------------- 

Generating response for gpt-4o model...
input tokens: 35
reasoning tokens: 0
output tokens: 29

 ---------------- 

testing for message: code, make sure you have installed the OpenAI Python
Generating response for o3-mini model...
input tokens: 17
reasoning tokens: 320
output tokens: 326

 ---------------- 

Generating response for o1 model...
input tokens: 17
reasoning tokens: 320
output tokens: 434

 ---------------- 

Generating response for o1-mini model...
input tokens: 19
reasoning tokens: 448
output tokens: 622

 ---------------- 

Generating response for o1-preview model...
input tokens: 19
reasoning tokens: 896
output tokens: 403

 ---------------- 

Generating response for gpt-4o model...
input tokens: 18
reasoning tokens: 0
output tokens: 428

 ----------------

notice that o3-mini and o1 have same input tokens, o1-preview and o1-mini have same and then we have gpt-4o. And all of them are offset by few tokens. I also did a similar test with a system prompt and there the difference was about 9 tokens, but always predictable.

@fast-blender-add-ons-dot-com
Copy link

This is a pretty famous PIP library for tons of people why don't you just go through in the code and explicitly define parameters for every open a I model and the second there's news that open a I released a new model just find out the pricing and update your library.... If you're really busy maybe you could research on GPT how to advertise or promote your other interests along the way to make that possible... Using your famous Github page

@fast-blender-add-ons-dot-com
Copy link

But in response to the other guy 03 mini and o1-mini don't have the same tokens it seems weird To have to sit here and do math... when it's easy to implement into the library but 03 mini and o1-mini have the same tokens input and output.... I mean there's this complex library I don't know all the parameters of it but it seems like there should be somewhere where you could explicitly define that 03 mini has $1.10 for input tokens and $4.40 for output tokens and just make a dictionary based on that and then Design your code around that dictionary so whenever there's like a new model or they change the prices then you just gotta update the price in the dictionary and then the code would just use that...

@Nicba1010
Copy link
Author

This is a pretty famous PIP library for tons of people why don't you just go through in the code and explicitly define parameters for every open a I model and the second there's news that open a I released a new model just find out the pricing and update your library.... If you're really busy maybe you could research on GPT how to advertise or promote your other interests along the way to make that possible... Using your famous Github page

Is that supposed to be in response to me? If so, I have no idea what this word salad is supposed to convey.

@fast-blender-add-ons-dot-com
Copy link

OKI just took 5 minutes to do the math on the function and it appears when I give it the oh one mini model it outputs the same price as the 03 mini model so I'm just going to make all my code whenever he needs to calculate 03 many tokens use the OH-1 mini model parameter

@fast-blender-add-ons-dot-com
Copy link

@hauntsaninja
Copy link
Collaborator

This is resolved in tiktoken 0.9

@fast-blender-add-ons-dot-com
Copy link

@tarekgh
Copy link

tarekgh commented Feb 14, 2025

adding the link of mapping the model's name to the encoding file just for clarity

tiktoken/tiktoken/model.py

Line 8 in e35ab09

"o1-": "o200k_base",
.

@fast-blender-add-ons-dot-com
Copy link

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
5 participants
@Nicba1010 @tarekgh @hauntsaninja @kashankrm @fast-blender-add-ons-dot-com