feat(core): Add byte size limit and oldest first truncation for gen_ai messages #17863
Conversation
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
RulaKhaled
changed the title
| // For models.generateContent: ContentListUnion: Content | Content[] | PartUnion | PartUnion[] | ||
| if ('contents' in params) { | ||
| span.setAttributes({ [GEN_AI_REQUEST_MESSAGES_ATTRIBUTE]: JSON.stringify(params.contents) }); | ||
| const contents = params.contents; |
There was a problem hiding this comment.
can you revert the comment removal to help others understand the request structure? this could also be a string
| } | ||
|
|
||
| // Extract and record AI request inputs, if present. This is intentionally separate from response attributes. | ||
| function addRequestAttributes(span: Span, params: Record<string, unknown>): void { |
There was a problem hiding this comment.
can you also revert back this JSDoc comment?
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
naaa760
force-pushed
the
genai-msg-limit
branch
from
6d97149 to
50d1d6d
Compare
|
@RulaKhaled |
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
There was a problem hiding this comment.
Thanks for resolving comments, it already looks better! I've pushed a quick commit for truncation functionality + added some JSDocs here :4d047e5 (this is a different branch) could you give this a shot? :) + can you take care of tests and the quick comments left?
naaa760
force-pushed
the
genai-msg-limit
branch
from
3ab7059 to
c91de63
Compare
|
@RulaKhaled |
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
I pushed changes to truncation file to help get this to finish line, but i'm having issues pushing more from my local setup to this branch. Can you resolve the rest of unresolved comments please? to be clear:
I'll pick this up again on Monday, thanks for resolving the comments :) |
Description:
fixes: #17809
gen_ai.request.messagesandvercel.ai.promptto prevent oversized payloads from exceeding ingestion limits.Changes:
messageTruncation.tsutility with byte size calculation and oldest-first truncation