Memory leak when using with_options() to configure options per request

[jose-speak]

Confirm this is an issue with the Python library and not an underlying OpenAI API

• This is an issue with the Python library

Describe the bug

Using with_options() to configure options per request introduces a memory leak.

How to determine it's a memory leak?

The following code calls the completion API 10 times. On each iteration, it takes a snapshot of traces of memory blocks using tracemalloc, compares it to the snapshot of the previous iteration, and prints the top 2 differences. The memory required by the openai library increases on every iteration. In particular, the following lines have the most increases (using sync client):

.../env/lib/python3.11/site-packages/openai/_response.py:227: size=321 KiB (+26.0 KiB), count=2681 (+175), average=123 B
.../env/lib/python3.11/site-packages/openai/_response.py:226: size=214 KiB (+19.4 KiB), count=1830 (+165), average=120 B

import os
import tracemalloc
from openai import OpenAI

tracemalloc.start()

client = OpenAI(api_key=os.getenv("OPENAI_API_KEY"))

snapshot1 = tracemalloc.take_snapshot()
for _ in range(10):
    client.with_options(max_retries=5).chat.completions.create(
        messages=[
            {
                "role": "user",
                "content": "How can I get the name of the current day in Node.js?",
            }
        ],
        model="gpt-3.5-turbo",
    )

    snapshot2 = tracemalloc.take_snapshot()
    top_stats = snapshot2.compare_to(snapshot1, 'lineno')
    print("[ Top 2 differences ]")
    for stat in top_stats[:2]:
        print(stat)
    
    snapshot1 = snapshot2

Potential cause

Looking at the library code, when using with_options(), a new client is created on every request (sync client and async client).

To Reproduce

Use with_options() and make multiple API calls.

Code snippets

No response

OS

macOS

Python version

Python v3.11.5

Library version

openai v1.3.3

[rattrayalex]

Thanks for reporting and sharing your debugging steps, we'll look into this soon!