Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

OpenAI: Ignore chunks with an empty choices list #192951

Merged
merged 2 commits into from
Sep 15, 2024
Merged

OpenAI: Ignore chunks with an empty choices list #192951

merged 2 commits into from
Sep 15, 2024

Conversation

pgayvallet
Copy link
Contributor

@pgayvallet pgayvallet commented Sep 14, 2024 •
edited by kibanamachine
Loading

Summary

OpenAI added usage stats to their APIs recently: https://community.openai.com/t/usage-stats-now-available-when-using-streaming-with-the-chat-completions-api-or-completions-api/738156

However, it's somewhat non BWC:

Note that this usage-specific chunk will have choices: [], so if you turn this feature on, you may need to update any code that accesses

(Héhéhé)

Leveraging this feature is for another day. This PR is just about avoiding the whole thing to explode.

@pgayvallet pgayvallet marked this pull request as ready for review September 14, 2024 19:17
@pgayvallet pgayvallet requested review from a team as code owners September 14, 2024 19:17
@pgayvallet
Copy link
Contributor Author

/ci

@pgayvallet pgayvallet added release_note:skip Skip the PR/issue when compiling release notes v9.0.0 backport:prev-minor Backport to the previous minor version (i.e. one version back from main) v8.16.0 Team:AI Infra AppEx AI Infrastructure Team and removed Team:Obs AI Assistant ci:project-deploy-observability Create an Observability project labels Sep 14, 2024
@elasticmachine
Copy link
Contributor

Pinging @elastic/appex-ai-infra (Team:AI Infra)

@pgayvallet pgayvallet enabled auto-merge (squash) September 15, 2024 15:46
@kibana-ci
Copy link
Collaborator

kibana-ci commented Sep 15, 2024 •
edited
Loading

💛 Build succeeded, but was flaky

  • Buildkite Build
  • Commit: e9bd958
  • Kibana Serverless Image: docker.elastic.co/kibana-ci/kibana-serverless:pr-192951-e9bd9586b3fc

Failed CI Steps

Metrics [docs]

✅ unchanged

History

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

@pgayvallet pgayvallet merged commit 8bffd61 into elastic:main Sep 15, 2024
24 checks passed
kibanamachine pushed a commit to kibanamachine/kibana that referenced this pull request Sep 15, 2024
@pgayvallet
OpenAI: Ignore chunks with an empty choices list (elastic#192951)
c598d8f 
## Summary

OpenAI added usage stats to their APIs recently:
https://community.openai.com/t/usage-stats-now-available-when-using-streaming-with-the-chat-completions-api-or-completions-api/738156

However, it's somewhat non BWC:

> Note that this usage-specific chunk will have choices: [], so if you
turn this feature on, you may need to update any code that accesses

(Héhéhé)

Leveraging this feature is for another day. This PR is just about
avoiding the whole thing to explode.

(cherry picked from commit 8bffd61)
@kibanamachine kibanamachine mentioned this pull request Sep 15, 2024
Merged
@kibanamachine
Copy link
Contributor

💚 All backports created successfully

Status Branch Result
✅ 8.x

Note: Successful backport PRs will be merged automatically after passing CI.

Questions ?

Please refer to the Backport tool documentation

This was referenced Sep 15, 2024
Merged
Open
kibanamachine added a commit that referenced this pull request Sep 15, 2024
@kibanamachine @pgayvallet
[8.x] OpenAI: Ignore chunks with an empty `choices` list (#…
5f8e564 
…192951) (#192960)

# Backport

This will backport the following commits from `main` to `8.x`:
- [OpenAI: Ignore chunks with an empty `choices` list
(#192951)](#192951)

<!--- Backport version: 9.4.3 -->

### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sqren/backport)

<!--BACKPORT [{"author":{"name":"Pierre
Gayvallet","email":"pierre.gayvallet@elastic.co"},"sourceCommit":{"committedDate":"2024-09-15T18:28:44Z","message":"OpenAI:
Ignore chunks with an empty `choices` list (#192951)\n\n##
Summary\r\n\r\nOpenAI added usage stats to their APIs
recently:\r\nhttps://community.openai.com/t/usage-stats-now-available-when-using-streaming-with-the-chat-completions-api-or-completions-api/738156\r\n\r\nHowever,
it's somewhat non BWC:\r\n\r\n> Note that this usage-specific chunk will
have choices: [], so if you\r\nturn this feature on, you may need to
update any code that accesses\r\n\r\n(Héhéhé)\r\n\r\nLeveraging this
feature is for another day. This PR is just about\r\navoiding the whole
thing to
explode.","sha":"8bffd618059aacc30d6190a0d143d8b0c7217faf","branchLabelMapping":{"^v9.0.0$":"main","^v8.16.0$":"8.x","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","v9.0.0","backport:prev-minor","Team:Obs
AI Assistant","ci:project-deploy-observability","v8.16.0","Team:AI
Infra"],"title":"OpenAI: Ignore chunks with an empty `choices`
list","number":192951,"url":"#192951:
Ignore chunks with an empty `choices` list (#192951)\n\n##
Summary\r\n\r\nOpenAI added usage stats to their APIs
recently:\r\nhttps://community.openai.com/t/usage-stats-now-available-when-using-streaming-with-the-chat-completions-api-or-completions-api/738156\r\n\r\nHowever,
it's somewhat non BWC:\r\n\r\n> Note that this usage-specific chunk will
have choices: [], so if you\r\nturn this feature on, you may need to
update any code that accesses\r\n\r\n(Héhéhé)\r\n\r\nLeveraging this
feature is for another day. This PR is just about\r\navoiding the whole
thing to
explode.","sha":"8bffd618059aacc30d6190a0d143d8b0c7217faf"}},"sourceBranch":"main","suggestedTargetBranches":["8.x"],"targetPullRequestStates":[{"branch":"main","label":"v9.0.0","branchLabelMappingKey":"^v9.0.0$","isSourceBranch":true,"state":"MERGED","url":"#192951:
Ignore chunks with an empty `choices` list (#192951)\n\n##
Summary\r\n\r\nOpenAI added usage stats to their APIs
recently:\r\nhttps://community.openai.com/t/usage-stats-now-available-when-using-streaming-with-the-chat-completions-api-or-completions-api/738156\r\n\r\nHowever,
it's somewhat non BWC:\r\n\r\n> Note that this usage-specific chunk will
have choices: [], so if you\r\nturn this feature on, you may need to
update any code that accesses\r\n\r\n(Héhéhé)\r\n\r\nLeveraging this
feature is for another day. This PR is just about\r\navoiding the whole
thing to
explode.","sha":"8bffd618059aacc30d6190a0d143d8b0c7217faf"}},{"branch":"8.x","label":"v8.16.0","branchLabelMappingKey":"^v8.16.0$","isSourceBranch":false,"state":"NOT_CREATED"}]}]
BACKPORT-->

Co-authored-by: Pierre Gayvallet <pierre.gayvallet@elastic.co>
pgayvallet added a commit that referenced this pull request Sep 16, 2024
@pgayvallet
OpenAI: Ignore chunks with an empty choices list (bis) (#192961)
08b682f 
## Summary

Follow-up of #192951 to address
the missing bits (given I don't know how to grep)
kibanamachine pushed a commit to kibanamachine/kibana that referenced this pull request Sep 16, 2024
@pgayvallet
OpenAI: Ignore chunks with an empty choices list (bis) (elastic#192961
b3ea9bf 
)

## Summary

Follow-up of elastic#192951 to address
the missing bits (given I don't know how to grep)

(cherry picked from commit 08b682f)
kibanamachine added a commit that referenced this pull request Sep 16, 2024
@kibanamachine @pgayvallet
[8.x] OpenAI: Ignore chunks with an empty &#x60;choices&#x60; list (b…
949f650 
…is) (#192961) (#192978)

# Backport

This will backport the following commits from `main` to `8.x`:
- [OpenAI: Ignore chunks with an empty &#x60;choices&#x60; list (bis)
(#192961)](#192961)

<!--- Backport version: 9.4.3 -->

### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sqren/backport)

<!--BACKPORT [{"author":{"name":"Pierre
Gayvallet","email":"pierre.gayvallet@elastic.co"},"sourceCommit":{"committedDate":"2024-09-16T09:06:54Z","message":"OpenAI:
Ignore chunks with an empty `choices` list (bis) (#192961)\n\n##
Summary\r\n\r\nFollow-up of
#192951 to address\r\nthe missing
bits (given I don't know how to
grep)","sha":"08b682fbd5f78ac82b42ddc7109e79f89e4ed961","branchLabelMapping":{"^v9.0.0$":"main","^v8.16.0$":"8.x","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","v9.0.0","backport:prev-minor","Team:Obs
AI Assistant","ci:project-deploy-observability","v8.16.0","Team:AI
Infra"],"title":"OpenAI: Ignore chunks with an empty `choices` list
(bis)","number":192961,"url":"#192961:
Ignore chunks with an empty `choices` list (bis) (#192961)\n\n##
Summary\r\n\r\nFollow-up of
#192951 to address\r\nthe missing
bits (given I don't know how to
grep)","sha":"08b682fbd5f78ac82b42ddc7109e79f89e4ed961"}},"sourceBranch":"main","suggestedTargetBranches":["8.x"],"targetPullRequestStates":[{"branch":"main","label":"v9.0.0","branchLabelMappingKey":"^v9.0.0$","isSourceBranch":true,"state":"MERGED","url":"#192961:
Ignore chunks with an empty `choices` list (bis) (#192961)\n\n##
Summary\r\n\r\nFollow-up of
#192951 to address\r\nthe missing
bits (given I don't know how to
grep)","sha":"08b682fbd5f78ac82b42ddc7109e79f89e4ed961"}},{"branch":"8.x","label":"v8.16.0","branchLabelMappingKey":"^v8.16.0$","isSourceBranch":false,"state":"NOT_CREATED"}]}]
BACKPORT-->

Co-authored-by: Pierre Gayvallet <pierre.gayvallet@elastic.co>
markov00 pushed a commit to markov00/kibana that referenced this pull request Sep 18, 2024
@pgayvallet @markov00
OpenAI: Ignore chunks with an empty choices list (elastic#192951)
b28cb4b 
## Summary

OpenAI added usage stats to their APIs recently:
https://community.openai.com/t/usage-stats-now-available-when-using-streaming-with-the-chat-completions-api-or-completions-api/738156

However, it's somewhat non BWC:

> Note that this usage-specific chunk will have choices: [], so if you
turn this feature on, you may need to update any code that accesses

(Héhéhé)

Leveraging this feature is for another day. This PR is just about
avoiding the whole thing to explode.
markov00 pushed a commit to markov00/kibana that referenced this pull request Sep 18, 2024
@pgayvallet @markov00
OpenAI: Ignore chunks with an empty choices list (bis) (elastic#192961
12c8524 
)

## Summary

Follow-up of elastic#192951 to address
the missing bits (given I don't know how to grep)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@dgieselaar dgieselaar dgieselaar approved these changes

@kobelb kobelb kobelb approved these changes

Assignees
No one assigned
Labels
backport:prev-minor Backport to the previous minor version (i.e. one version back from main) ci:project-deploy-observability Create an Observability project release_note:skip Skip the PR/issue when compiling release notes Team:AI Infra AppEx AI Infrastructure Team Team:Obs AI Assistant v8.16.0 v9.0.0
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
6 participants
@pgayvallet @elasticmachine @kibana-ci @kibanamachine @dgieselaar @kobelb