Is upload_file thread safe? #327

jiupinjiandingshi · 2024-05-08T08:55:53Z

Description of the bug:

when I want use genai , I need to use this method in an interface request, but when I put "genai.upload_file(path=path,display_name=file)" into the interface method, it will report an error SLL_ERROR
@app.route("/xxx",method=["GET"]) def upload_video(): output_frame_folder = requests.output_frame_folder time.sleep(1.0) lists = [] try: for file in os.listdir(f"{output_frame_folder}"): path = os.path.join(f"{output_frame_folder}", file) sample_file = genai.upload_file(path=path,display_name=file)

Actual vs expected behavior:

No response

Any other information you'd like to share?

No response

The text was updated successfully, but these errors were encountered:

jiupinjiandingshi · 2024-05-08T08:58:14Z

Does this mean that this method does not support concurrency?

jiupinjiandingshi · 2024-05-08T09:02:11Z

the error desc:[SSL: WRONG_VERSION_NUMBER] wrong version number (_ssl.c:2621)

singhniraj08 · 2024-05-08T09:53:31Z

@jiupinjiandingshi, I tried running Prompting with media files tutorial and genai.upload_file works without any issues. Can you please share a colab gist or code which we can try to reproduce on our end. Thank you!

jiupinjiandingshi · 2024-05-08T10:18:19Z

If I run it once, there is no problem, as follows：
for file in os.listdir(f"{output_frame_folder}"): path = os.path.join(f"{output_frame_folder}", file) sample_file = genai.upload_file(path=path,display_name=file)

But if my interface receives two requests at the same time, this code may be called twice in the same time period. In this case, an error will be reported as follows:
httplib2.Http() error:[SSL: WRONG_VERSION_NUMBER] wrong version number (_ssl.c:2621)
pit:’output_frame_folder ‘ It is a folder that contains some pictures and an MP3

sMx7d · 2024-05-22T19:35:12Z

try to run pip install google-generativeai --upgrade that solved it for me!
i was using google-generativeai-0.3.2 but now its google-ai-generativelanguage-0.6.4

MarkDaoust · 2024-05-22T20:25:13Z

Thanks, please reopen this if the problem comes back in a reproducible way.

ronfromhp · 2024-05-23T02:10:59Z

i tried upgrading the library as suggested by @sMx7d . But still getting errors

 File "/code/vision_models.py", line 228, in media2concept
video-backend  |     messages = [
video-backend  |                ^
video-backend  |   File "/code/vision_models.py", line 231, in <listcomp>
video-backend  |     "parts": await asyncio.create_task(make_video_prompt(media))
video-backend  |              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
video-backend  |   File "/code/vision_models.py", line 154, in make_video_prompt
video-backend  |     framefiles = await upload_frames_concurrently(framefiles)
video-backend  |                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
video-backend  |   File "/code/vision_models.py", line 136, in upload_frames_concurrently
video-backend  |     uploaded_frames = await asyncio.gather(*tasks)
video-backend  |                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
video-backend  |   File "/code/vision_models.py", line 127, in aupload_frame
video-backend  |     res = await asyncio.to_thread(genai.upload_file, ff.file_path, name=ff.name)
video-backend  |           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
video-backend  |   File "/usr/local/lib/python3.11/asyncio/threads.py", line 25, in to_thread
video-backend  |     return await loop.run_in_executor(None, func_call)
video-backend  |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
video-backend  |   File "/usr/local/lib/python3.11/concurrent/futures/thread.py", line 58, in run
video-backend  |     result = self.fn(*self.args, **self.kwargs)
video-backend  |              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
video-backend  |   File "/usr/local/lib/python3.11/site-packages/google/generativeai/files.py", line 69, in upload_file
video-backend  |     response = client.create_file(
video-backend  |                ^^^^^^^^^^^^^^^^^^^
video-backend  |   File "/usr/local/lib/python3.11/site-packages/google/generativeai/client.py", line 79, in create_file
video-backend  |     return self.get_file({"name": result["file"]["name"]})
video-backend  |                                   ~~~~~~^^^^^^^^
video-backend  | TypeError: string indices must be integers, not 'str'

anyone else finding the same issue?

sMx7d · 2024-05-23T03:02:28Z

@ronfromhp , uninstall and reinstall it.

ronfromhp · 2024-05-23T03:24:15Z

@sMx7d somehow its gone back to giving the ssl error

these are the installed libraries:
google-ai-generativelanguage==0.6.4
google-api-core==2.19.0
google-api-python-client==2.130.0
google-auth==2.29.0
google-auth-httplib2==0.2.0
google-generativeai==0.5.4
googleapis-common-protos==1.63.0
grpcio==1.64.0
grpcio-status==1.62.2
h11==0.14.0
httpcore==1.0.5
httplib2==0.22.0
httpx==0.27.0

This is the same old error trace im getting:

video-backend  |     uploaded_frames = await asyncio.gather(*tasks)
video-backend  |                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
video-backend  |   File "/code/vision_models.py", line 127, in aupload_frame
video-backend  |     res = await asyncio.to_thread(genai.upload_file, ff.file_path, name=ff.name)
video-backend  |           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
video-backend  |   File "/usr/local/lib/python3.11/asyncio/threads.py", line 25, in to_thread
video-backend  |     return await loop.run_in_executor(None, func_call)
video-backend  |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
video-backend  |   File "/usr/local/lib/python3.11/concurrent/futures/thread.py", line 58, in run
video-backend  |     result = self.fn(*self.args, **self.kwargs)
video-backend  |              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
video-backend  |   File "/usr/local/lib/python3.11/site-packages/google/generativeai/files.py", line 69, in upload_file
video-backend  |     response = client.create_file(
video-backend  |                ^^^^^^^^^^^^^^^^^^^
video-backend  |   File "/usr/local/lib/python3.11/site-packages/google/generativeai/client.py", line 77, in create_file
video-backend  |     result = request.execute()
video-backend  |              ^^^^^^^^^^^^^^^^^
video-backend  |   File "/usr/local/lib/python3.11/site-packages/googleapiclient/_helpers.py", line 130, in positional_wrapper
video-backend  |     return wrapped(*args, **kwargs)
video-backend  |            ^^^^^^^^^^^^^^^^^^^^^^^^
video-backend  |   File "/usr/local/lib/python3.11/site-packages/googleapiclient/http.py", line 902, in execute
video-backend  |     _, body = self.next_chunk(http=http, num_retries=num_retries)
video-backend  |               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
video-backend  |   File "/usr/local/lib/python3.11/site-packages/googleapiclient/_helpers.py", line 130, in positional_wrapper
video-backend  |     return wrapped(*args, **kwargs)
video-backend  |            ^^^^^^^^^^^^^^^^^^^^^^^^
video-backend  |   File "/usr/local/lib/python3.11/site-packages/googleapiclient/http.py", line 1007, in next_chunk
video-backend  |     resp, content = _retry_request(
video-backend  |                     ^^^^^^^^^^^^^^^
video-backend  |   File "/usr/local/lib/python3.11/site-packages/googleapiclient/http.py", line 222, in _retry_request
video-backend  |     raise exception
video-backend  |   File "/usr/local/lib/python3.11/site-packages/googleapiclient/http.py", line 191, in _retry_request
video-backend  |     resp, content = http.request(uri, method, *args, **kwargs)
video-backend  |                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
video-backend  |   File "/usr/local/lib/python3.11/site-packages/httplib2/__init__.py", line 1724, in request
video-backend  |     (response, content) = self._request(
video-backend  |                           ^^^^^^^^^^^^^^
video-backend  |   File "/usr/local/lib/python3.11/site-packages/httplib2/__init__.py", line 1444, in _request
video-backend  |     (response, content) = self._conn_request(conn, request_uri, method, body, headers)
video-backend  |                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
video-backend  |   File "/usr/local/lib/python3.11/site-packages/httplib2/__init__.py", line 1396, in _conn_request
video-backend  |     response = conn.getresponse()
video-backend  |                ^^^^^^^^^^^^^^^^^^
video-backend  |   File "/usr/local/lib/python3.11/http/client.py", line 1395, in getresponse
video-backend  |     response.begin()
video-backend  |   File "/usr/local/lib/python3.11/http/client.py", line 325, in begin
video-backend  |     version, status, reason = self._read_status()
video-backend  |                               ^^^^^^^^^^^^^^^^^^^
video-backend  |   File "/usr/local/lib/python3.11/http/client.py", line 286, in _read_status
video-backend  |     line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
video-backend  |                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
video-backend  |   File "/usr/local/lib/python3.11/socket.py", line 706, in readinto
video-backend  |     return self._sock.recv_into(b)
video-backend  |            ^^^^^^^^^^^^^^^^^^^^^^^
video-backend  |   File "/usr/local/lib/python3.11/ssl.py", line 1314, in recv_into
video-backend  |     return self.read(nbytes, buffer)
video-backend  |            ^^^^^^^^^^^^^^^^^^^^^^^^^
video-backend  |   File "/usr/local/lib/python3.11/ssl.py", line 1166, in read
video-backend  |     return self._sslobj.read(len, buffer)
video-backend  |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
video-backend  | ssl.SSLError: [SSL: WRONG_VERSION_NUMBER] wrong version number (_ssl.c:2580)

sMx7d · 2024-05-23T03:28:37Z

Its stil a ssl error , sorry but i don't know anything in ssl or what it means .
But it seems that genertiveai version is 0.5.4 , IDK if it has upload file function or not , but either way, i don't think upgrading the library will solve as its a ssl error and not related to library functions.
I hope my response was clear, if you needed anything just mention me.
@ronfromhp

ronfromhp · 2024-05-23T14:45:49Z

@jiupinjiandingshi can you reopen the issue? As currently I havent found a resolution which works

sMx7d · 2024-05-24T18:49:17Z

video-backend  | TypeError: string indices must be integers, not 'str'

sorry , i just reviewed the error and i saw video-backend | TypeError: string indices must be integers, not 'str'
that error is likely not from the generative-ai library , its from your code but IDK what the solve for it.

ronfromhp · 2024-05-25T19:58:11Z

as of my previous comment i was still getting the same ssl error even after updating genai @sMx7d

MarkDaoust · 2024-10-02T22:43:48Z

Yeah, this error is reproducible, sorry I missed the follow ups here after the initial close.

This script works fine:

https://gist.github.com/MarkDaoust/dcd65b626bf4683860aa510b79bc225e

But if I add an upload_file to the code that runs in the threads I get a messy SSL error:

https://gist.github.com/MarkDaoust/73035a173d532f03c1a2c5aa49de9993

Traceback (most recent call last):
  File "/Users/markdaoust/Projects/generative-ai-python/run_upload_threads.py", line 164, in <module>
    for n, (prompt, response) in enumerate(zip(prompts, responses)):
                                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/markdaoust/homebrew/Cellar/python@3.12/3.12.5/Frameworks/Python.framework/Versions/3.12/lib/python3.12/concurrent/futures/_base.py", line 619, in result_iterator
    yield _result_or_cancel(fs.pop())
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/markdaoust/homebrew/Cellar/python@3.12/3.12.5/Frameworks/Python.framework/Versions/3.12/lib/python3.12/concurrent/futures/_base.py", line 317, in _result_or_cancel
    return fut.result(timeout)
           ^^^^^^^^^^^^^^^^^^^
  File "/Users/markdaoust/homebrew/Cellar/python@3.12/3.12.5/Frameworks/Python.framework/Versions/3.12/lib/python3.12/concurrent/futures/_base.py", line 456, in result
    return self.__get_result()
           ^^^^^^^^^^^^^^^^^^^
  File "/Users/markdaoust/homebrew/Cellar/python@3.12/3.12.5/Frameworks/Python.framework/Versions/3.12/lib/python3.12/concurrent/futures/_base.py", line 401, in __get_result
    raise self._exception
  File "/Users/markdaoust/homebrew/Cellar/python@3.12/3.12.5/Frameworks/Python.framework/Versions/3.12/lib/python3.12/concurrent/futures/thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/markdaoust/Projects/generative-ai-python/run_upload_threads.py", line 157, in get_result
    f = genai.upload_file(fpath)
        ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/markdaoust/Projects/generative-ai-python/google/generativeai/files.py", line 85, in upload_file
    response = client.create_file(
               ^^^^^^^^^^^^^^^^^^^
  File "/Users/markdaoust/Projects/generative-ai-python/google/generativeai/client.py", line 121, in create_file
    result = request.execute()
             ^^^^^^^^^^^^^^^^^
  File "/Users/markdaoust/Projects/venv3/lib/python3.12/site-packages/googleapiclient/_helpers.py", line 130, in positional_wrapper
    return wrapped(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/markdaoust/Projects/venv3/lib/python3.12/site-packages/googleapiclient/http.py", line 902, in execute
    _, body = self.next_chunk(http=http, num_retries=num_retries)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/markdaoust/Projects/venv3/lib/python3.12/site-packages/googleapiclient/_helpers.py", line 130, in positional_wrapper
    return wrapped(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/markdaoust/Projects/venv3/lib/python3.12/site-packages/googleapiclient/http.py", line 1084, in next_chunk
    resp, content = http.request(
                    ^^^^^^^^^^^^^
  File "/Users/markdaoust/Projects/venv3/lib/python3.12/site-packages/httplib2/__init__.py", line 1724, in request
    (response, content) = self._request(
                          ^^^^^^^^^^^^^^
  File "/Users/markdaoust/Projects/venv3/lib/python3.12/site-packages/httplib2/__init__.py", line 1444, in _request
    (response, content) = self._conn_request(conn, request_uri, method, body, headers)
                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/markdaoust/Projects/venv3/lib/python3.12/site-packages/httplib2/__init__.py", line 1396, in _conn_request
    response = conn.getresponse()
               ^^^^^^^^^^^^^^^^^^
  File "/Users/markdaoust/homebrew/Cellar/python@3.12/3.12.5/Frameworks/Python.framework/Versions/3.12/lib/python3.12/http/client.py", line 1428, in getresponse
    response.begin()
  File "/Users/markdaoust/homebrew/Cellar/python@3.12/3.12.5/Frameworks/Python.framework/Versions/3.12/lib/python3.12/http/client.py", line 331, in begin
    version, status, reason = self._read_status()
                              ^^^^^^^^^^^^^^^^^^^
  File "/Users/markdaoust/homebrew/Cellar/python@3.12/3.12.5/Frameworks/Python.framework/Versions/3.12/lib/python3.12/http/client.py", line 292, in _read_status
    line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/markdaoust/homebrew/Cellar/python@3.12/3.12.5/Frameworks/Python.framework/Versions/3.12/lib/python3.12/socket.py", line 720, in readinto
    return self._sock.recv_into(b)
           ^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/markdaoust/homebrew/Cellar/python@3.12/3.12.5/Frameworks/Python.framework/Versions/3.12/lib/python3.12/ssl.py", line 1252, in recv_into
    return self.read(nbytes, buffer)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/markdaoust/homebrew/Cellar/python@3.12/3.12.5/Frameworks/Python.framework/Versions/3.12/lib/python3.12/ssl.py", line 1104, in read
    return self._sslobj.read(len, buffer)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ssl.SSLError: [SSL] record layer failure (_ssl.c:2559)

MarkDaoust · 2024-10-02T23:05:28Z

Upload file is one method that isn't auto-generated in the google.ai.generativelanguage client library.

I added code client.py#L69-L123 to implement the API's create_file, since the generated client doesn't handle it.

This code is based on the discovery API, and I shouldn't be sharing those discovery instances between threads:

Thanks @jachor for pointing this out.

So I think we just need to make discovery_api thread local here:

https://github.com/google-gemini/generative-ai-python/blob/main/google/generativeai/client.py#L86-L88

Also related: #564

sumeet-desai · 2024-10-03T10:08:53Z

class FileServiceClient(glm.FileServiceClient):
    def __init__(self, *args, **kwargs):
        self._local = threading.local()
        super().__init__(*args, **kwargs)

    def _setup_discovery_api(self, metadata: dict | Sequence[tuple[str, str]] = ()):
        if not hasattr(self._local, 'discovery_api'):
            api_key = self._client_options.api_key
            if api_key is None:
                raise ValueError(
                    "Invalid operation: Uploading to the File API requires an API key. Please provide a valid API key."
                )

            request = googleapiclient.http.HttpRequest(
                http=httplib2.Http(),
                postproc=lambda resp, content: (resp, content),
                uri=f"{GENAI_API_DISCOVERY_URL}?version=v1beta&key={api_key}",
                headers=dict(metadata),
            )
            response, content = request.execute()
            request.http.close()

            discovery_doc = content.decode("utf-8")
            self._local.discovery_api = googleapiclient.discovery.build_from_document(
                discovery_doc, developerKey=api_key
            )

    def create_file(
        self,
        path: str | pathlib.Path | os.PathLike,
        *,
        mime_type: str | None = None,
        name: str | None = None,
        display_name: str | None = None,
        resumable: bool = True,
        metadata: Sequence[tuple[str, str]] = (),
    ) -> protos.File:
        self._setup_discovery_api(metadata)

        file = {}
        if name is not None:
            file["name"] = name
        if display_name is not None:
            file["displayName"] = display_name

        media = googleapiclient.http.MediaFileUpload(
            filename=path, mimetype=mime_type, resumable=resumable
        )
        request = self._local.discovery_api.media().upload(body={"file": file}, media_body=media)
        for key, value in metadata:
            request.headers[key] = value
        result = request.execute()

        return self.get_file({"name": result["file"]["name"]})

@MarkDaoust this is working for me with your suggestion for anyones reference

MarkDaoust · 2024-10-03T18:13:16Z

@sumeet-desai, Nice. That looks right. Can you send a PR?

sumeet-desai · 2024-10-05T16:03:04Z

@MarkDaoust I've raised it here
#583

wkevwang · 2024-10-29T21:41:49Z

Experiencing this too - excited for the update.

gokturkDev · 2024-11-06T08:54:11Z

hi any updates on this its a real bugger for prod apps. When will it be released?

jiupinjiandingshi added component:python sdk Issue/PR related to Python SDK type:bug Something isn't working labels May 8, 2024

singhniraj08 added the status:awaiting user response Awaiting a response from the author label May 8, 2024

MarkDaoust changed the title ~~i can't use genai.upload_file(path=path,display_name=file)~~ Is upload_file thread safe? May 10, 2024

singhniraj08 assigned MarkDaoust May 10, 2024

singhniraj08 added status:triaged Issue/PR triaged to the corresponding sub-team and removed status:awaiting user response Awaiting a response from the author labels May 10, 2024

MarkDaoust closed this as completed May 22, 2024

github-actions bot removed the status:triaged Issue/PR triaged to the corresponding sub-team label May 22, 2024

MarkDaoust mentioned this issue Oct 2, 2024

Docs fail to document thread-safety of genai client, and it fails irrecoverably on multi-threaded use #211

Closed

MarkDaoust reopened this Oct 2, 2024

MarkDaoust mentioned this issue Oct 2, 2024

Asynchronous file upload (google.generativeai.upload_file) and other blocking methods? #564

Open

Sumeet213 mentioned this issue Oct 3, 2024

fix: thread-safe discovery API setup (Issues#327) #582

Closed

6 tasks

Sumeet213 mentioned this issue Oct 3, 2024

fix: thread-safe discovery API setup (Issues#327) #583

Merged

6 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Is upload_file thread safe? #327

Is upload_file thread safe? #327

jiupinjiandingshi commented May 8, 2024

jiupinjiandingshi commented May 8, 2024

jiupinjiandingshi commented May 8, 2024

singhniraj08 commented May 8, 2024

jiupinjiandingshi commented May 8, 2024

sMx7d commented May 22, 2024

MarkDaoust commented May 22, 2024

ronfromhp commented May 23, 2024

sMx7d commented May 23, 2024

ronfromhp commented May 23, 2024

sMx7d commented May 23, 2024

ronfromhp commented May 23, 2024

sMx7d commented May 24, 2024

ronfromhp commented May 25, 2024

MarkDaoust commented Oct 2, 2024 •

edited

Loading

MarkDaoust commented Oct 2, 2024 •

edited

Loading

sumeet-desai commented Oct 3, 2024 •

edited

Loading

MarkDaoust commented Oct 3, 2024

sumeet-desai commented Oct 5, 2024

wkevwang commented Oct 29, 2024

gokturkDev commented Nov 6, 2024 •

edited

Loading

Is upload_file thread safe? #327

Is upload_file thread safe? #327

Comments

jiupinjiandingshi commented May 8, 2024

Description of the bug:

Actual vs expected behavior:

Any other information you'd like to share?

jiupinjiandingshi commented May 8, 2024

jiupinjiandingshi commented May 8, 2024

singhniraj08 commented May 8, 2024

jiupinjiandingshi commented May 8, 2024

sMx7d commented May 22, 2024

MarkDaoust commented May 22, 2024

ronfromhp commented May 23, 2024

sMx7d commented May 23, 2024

ronfromhp commented May 23, 2024

sMx7d commented May 23, 2024

ronfromhp commented May 23, 2024

sMx7d commented May 24, 2024

ronfromhp commented May 25, 2024

MarkDaoust commented Oct 2, 2024 • edited Loading

MarkDaoust commented Oct 2, 2024 • edited Loading

sumeet-desai commented Oct 3, 2024 • edited Loading

MarkDaoust commented Oct 3, 2024

sumeet-desai commented Oct 5, 2024

wkevwang commented Oct 29, 2024

gokturkDev commented Nov 6, 2024 • edited Loading

MarkDaoust commented Oct 2, 2024 •

edited

Loading

MarkDaoust commented Oct 2, 2024 •

edited

Loading

sumeet-desai commented Oct 3, 2024 •

edited

Loading

gokturkDev commented Nov 6, 2024 •

edited

Loading