gemini using real sse format response #3677 #3688 #4906
Conversation
|
@lloydzhou is attempting to deploy a commit to the NextChat Team on Vercel. A member of the Team first needs to authorize it. |
WalkthroughThe changes primarily introduce conditional URL modifications and enhanced response handling for streaming data through the refactoring of the Changes
Poem
Tip Early access features: enabledWe are currently testing the following features in early access:
Note:
Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media? TipsChatThere are 3 ways to chat with CodeRabbit:
Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. CodeRabbit Commands (invoked as PR comments)
Additionally, you can add CodeRabbit Configration File (
|
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 0
Outside diff range and nitpick comments (1)
app/client/platforms/google.ts (1)
Line range hint
150-262: Streaming response handling looks good. Consider optimizinganimateResponseText.The updates to handle streaming responses using
fetchEventSourceare necessary and well-implemented. However, theanimateResponseTextfunction could be optimized to reduce the number of calls torequestAnimationFrame.- const fetchCount = Math.max(1, Math.round(remainText.length / 60)); - const fetchText = remainText.slice(0, fetchCount); - responseText += fetchText; - remainText = remainText.slice(fetchCount); - options.onUpdate?.(responseText, fetchText); + const fetchText = remainText.slice(0, 60); + responseText += fetchText; + remainText = remainText.slice(60); + options.onUpdate?.(responseText, fetchText);
Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Files selected for processing (2)
- app/api/google/[...path]/route.ts (1 hunks)
- app/client/platforms/google.ts (6 hunks)
Additional comments not posted (4)
app/api/google/[...path]/route.ts (1)
66-68: LGTM! But verify the function usage in the codebase.The code changes are approved.
However, ensure that all function calls to
handlematch the new logic.Verification successful
LGTM! The function usage is correctly implemented.
The
fetchUrlconstruction logic matches the new logic, and the function handles the request and response appropriately. No further issues are found.
app/api/google/[...path]/route.ts: Verified thehandlefunction implementation.Scripts executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash # Description: Verify all function calls to `handle` match the new logic. # Test: Search for the function usage. Expect: Only occurrences of the new logic. rg --type js --type ts $'handle'Length of output: 1863
Script:
#!/bin/bash # Extract the implementation of the `handle` function from `app/api/google/[...path]/route.ts` ast-grep --lang typescript --pattern $'async function handle($_) { $$$ }' app/api/google/[...path]/route.tsLength of output: 112
Script:
#!/bin/bash # Search for the `handle` function definition within `app/api/google/[...path]/route.ts` rg 'async function handle' app/api/google/[...path]/route.ts -A 20Length of output: 576
Script:
#!/bin/bash # Extend the search to capture more lines of the `handle` function implementation rg 'async function handle' app/api/google/[...path]/route.ts -A 50Length of output: 1260
Script:
#!/bin/bash # Extend the search to capture the complete `handle` function implementation rg 'async function handle' app/api/google/[...path]/route.ts -A 100Length of output: 2518
app/client/platforms/google.ts (3)
6-11: Imports look good.The newly added imports are necessary for the updated functionality.
129-131: Base URL construction looks good.The updates to the
baseUrlconstruction handle custom configurations and app-specific URLs appropriately.
Line range hint
263-276: Non-streaming response handling looks good.The updates to differentiate between streaming and non-streaming responses are necessary and well-implemented.
|
Your build has completed! |
There was a problem hiding this comment.
Actionable comments posted: 1
Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Files selected for processing (1)
- app/client/platforms/google.ts (6 hunks)
Additional comments not posted (7)
app/client/platforms/google.ts (7)
150-150: Timeout Handling: Ensure proper cleanupEnsure that the request timeout is properly handled and cleaned up to avoid potential memory leaks or dangling requests.
Consider adding a comment or additional logic to ensure proper cleanup.
6-10: New Imports: Ensure correct usage and compatibilityNew imports have been added for
Locale,EventStreamContentType,fetchEventSource, andprettyObject. Ensure these imports are correctly used and compatible with the rest of the codebase.Verification successful
New Imports: Verified for correct usage and compatibility
The newly added imports
Locale,EventStreamContentType,fetchEventSource, andprettyObjectare correctly used and integrated across the codebase.
Localeis used in various components and utility files.EventStreamContentTypeandfetchEventSourceare used in multiple platform files.prettyObjectis utilized in different utility and platform files.Scripts executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash # Description: Verify the usage and compatibility of newly added imports. # Test: Search for the usage of `Locale`, `EventStreamContentType`, `fetchEventSource`, and `prettyObject`. Expect: Correct usage in the codebase. rg --type python $'Locale|EventStreamContentType|fetchEventSource|prettyObject'Length of output: 111
Script:
#!/bin/bash # Description: Verify the usage and compatibility of newly added imports. # Test: Search for the usage of `Locale`, `EventStreamContentType`, `fetchEventSource`, and `prettyObject`. Expect: Correct usage in the codebase. rg --type ts 'Locale|EventStreamContentType|fetchEventSource|prettyObject'Length of output: 30208
129-131: Conditional URL Construction: Verify correctnessThe construction of
baseUrlhas been updated for streaming requests. Ensure the URL is correctly constructed based on the environment and configuration.Verification successful
Conditional URL Construction: Verification of Correctness
The construction of
baseUrlinapp/client/platforms/google.tsappears to be correctly handled based on the environment and configuration. The URL is conditionally assigned based on whether custom configurations are used, and further modified based on whether the application is running in an app environment.
- Lines of Interest:
let baseUrl = "";if (accessStore.useCustomConfig) { baseUrl = accessStore.googleUrl; }if (!baseUrl) { baseUrl = isApp ? DEFAULT_API_HOST + "/api/proxy/google/" + Google.ChatPath(modelConfig.model) : this.path(Google.ChatPath(modelConfig.model)); }if (isApp) { baseUrl +=?key=${accessStore.googleApiKey}; }const chatPath = baseUrl.replace("generateContent", "streamGenerateContent") + (baseUrl.indexOf("?") > -1 ? "&alt=sse" : "?alt=sse");The URL construction logic ensures that
baseUrlis correctly set based on the environment (isApp) and configuration (useCustomConfig). Additionally, the URL is modified appropriately for streaming requests by replacinggenerateContentwithstreamGenerateContentand appending query parameters as needed.Scripts executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash # Description: Verify the correctness of URL construction for streaming requests. # Test: Search for the construction of `baseUrl`. Expect: Correct URL construction logic. rg --type python $'baseUrl'Length of output: 57
Script:
#!/bin/bash # Description: Verify the correctness of URL construction for streaming requests. # Test: Search for the construction of `baseUrl`. Expect: Correct URL construction logic. rg --type typescript 'baseUrl'Length of output: 65
Script:
#!/bin/bash # Description: Verify the correctness of URL construction for streaming requests. # Test: Search for the construction of `baseUrl`. Expect: Correct URL construction logic. rg 'baseUrl' --glob '*.ts'Length of output: 3914
Script:
#!/bin/bash # Description: Extract and review the relevant lines from app/client/platforms/google.ts to verify URL construction. # Extract lines around the baseUrl construction in app/client/platforms/google.ts rg -A 10 -B 10 'baseUrl' app/client/platforms/google.tsLength of output: 2529
189-261: Streaming Response Handling: Ensure proper handling and error managementThe logic for handling streaming responses using
fetchEventSourcehas been added. Ensure that the response handling, error management, and resource cleanup are properly implemented.Verification successful
Streaming Response Handling: Verified
The implementation of
fetchEventSourcein the filesgoogle.ts,openai.ts, andanthropic.tscorrectly handles streaming responses, manages errors, and ensures resource cleanup. The logic is consistent and includes all necessary components.
app/client/platforms/google.tsapp/client/platforms/openai.tsapp/client/platforms/anthropic.tsScripts executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash # Description: Verify the handling of streaming responses using `fetchEventSource`. # Test: Search for the usage of `fetchEventSource`. Expect: Proper handling of streaming responses, error management, and resource cleanup. rg --type python $'fetchEventSource'Length of output: 66
Script:
#!/bin/bash # Description: Verify the handling of streaming responses using `fetchEventSource`. # Test: Search for the usage of `fetchEventSource` in TypeScript files. Expect: Proper handling of streaming responses, error management, and resource cleanup. rg --type typescript 'fetchEventSource'Length of output: 74
Script:
#!/bin/bash # Description: Verify the handling of streaming responses using `fetchEventSource`. # Test: Search for the usage of `fetchEventSource` in TypeScript files. Expect: Proper handling of streaming responses, error management, and resource cleanup. rg --type ts 'fetchEventSource'Length of output: 390
Script:
#!/bin/bash # Description: Extract the usage context of `fetchEventSource` in relevant TypeScript files. # Extract and display the context around `fetchEventSource` in the identified files. rg --type ts -A 20 -B 10 'fetchEventSource' app/client/platforms/anthropic.ts app/client/platforms/openai.ts app/client/platforms/google.tsLength of output: 11342
185-188: Conditional SSE URL: Ensure correct URL constructionThe construction of the SSE URL is conditional based on the presence of query parameters. Ensure the URL is correctly constructed for both cases.
Verification successful
Conditional SSE URL: Ensure correct URL construction
The SSE URL construction logic correctly handles both cases of the presence and absence of query parameters.
- The URL is correctly constructed by replacing "generateContent" with "streamGenerateContent".
- It appends
&alt=sseif there are existing query parameters, or?alt=sseif there are none.Scripts executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash # Description: Verify the correctness of SSE URL construction. # Test: Search for the construction of the SSE URL. Expect: Correct URL construction logic for both cases. rg --type python $'streamGenerateContent'Length of output: 71
Script:
#!/bin/bash # Description: Verify the correctness of SSE URL construction. # Test: Search for the construction of the SSE URL. Expect: Correct URL construction logic for both cases. rg 'streamGenerateContent' --type typescriptLength of output: 79
Script:
#!/bin/bash # Description: Verify the correctness of SSE URL construction. # Test: Search for the construction of the SSE URL. Expect: Correct URL construction logic for both cases. rg 'streamGenerateContent' --type tsLength of output: 139
Script:
#!/bin/bash # Description: Retrieve the relevant lines of code around the SSE URL construction. # Test: Display the relevant lines of code around the SSE URL construction. rg 'baseUrl.replace("generateContent", "streamGenerateContent")' --context 5 --type tsLength of output: 88
Script:
#!/bin/bash # Description: Retrieve the relevant lines of code around the SSE URL construction. # Test: Display the relevant lines of code around the SSE URL construction. rg 'baseUrl.replace("generateContent", "streamGenerateContent")' app/client/platforms/google.ts -A 5 -B 5Length of output: 107
Script:
#!/bin/bash # Description: Retrieve the relevant lines of code around the SSE URL construction. # Test: Display the relevant lines of code around the SSE URL construction. sed -n '180,190p' app/client/platforms/google.tsLength of output: 477
275-275: Extracted Message: Verify correctnessThe line
const message = apiClient.extractMessage(resJson);has been updated. Ensure that the extracted message is correctly handled and used.Verification successful
Verified: Extracted Message Handling
The extracted message from
apiClient.extractMessage(resJson)is correctly handled and used in the code:
The extracted message is appended to
remainText.The extracted message is passed to the
options.onFinishcallback function.Scripts executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash # Description: Verify the correctness of the extracted message handling. # Test: Search for the usage of `apiClient.extractMessage`. Expect: Correct handling and usage of the extracted message. rg --type python $'extractMessage'Length of output: 64
Script:
#!/bin/bash # Description: Verify the correctness of the extracted message handling. # Test: Search for the usage of `apiClient.extractMessage`. Expect: Correct handling and usage of the extracted message. rg --type ts 'extractMessage'Length of output: 566
Script:
#!/bin/bash # Extract surrounding lines of the occurrences in google.ts to analyze the context and usage of the extracted message. rg --type ts -A 5 -B 5 'extractMessage' app/client/platforms/google.tsLength of output: 1025
183-184: Abort Handling: Ensure proper response completionEnsure that the
controller.signal.onabortproperly handles the response completion and cleans up resources.
https://github.com/google-gemini/cookbook/blob/main/quickstarts/rest/Streaming_REST.ipynb
Summary by CodeRabbit
New Features
Refactor