Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Added E2E LLM test #26914

Open
wants to merge 15 commits into
base: master
Choose a base branch
from
Open

Added E2E LLM test #26914

wants to merge 15 commits into from

Conversation

AlexKoff88
Copy link
Contributor

No description provided.

@AlexKoff88 AlexKoff88 requested a review from a team as a code owner October 4, 2024 09:35
@AlexKoff88 AlexKoff88 requested a review from a team as a code owner October 4, 2024 09:41
@AlexKoff88 AlexKoff88 requested review from akuporos and artanokhov and removed request for a team October 4, 2024 09:41
@github-actions github-actions bot added the category: dependency_changes Pull requests that update a dependency file label Oct 4, 2024
ilya-lavrenov
ilya-lavrenov reviewed Oct 7, 2024
View reviewed changes
Copy link
Contributor

@ilya-lavrenov ilya-lavrenov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I suppose the right place is GenAI repo?
There we will switch to nightly releases soon, so can perform integration tests E2E

@AlexKoff88
Copy link
Contributor Author

I suppose the right place is GenAI repo? There we will switch to nightly releases soon, so can perform integration tests E2E

The idea is to understand the status of PRs in terms of accuracy on LLMs. This test should work for this purpose and it is quick at the same time.
For GenAI, we have a nightly scope in E2E validation.

@AlexKoff88 AlexKoff88 changed the title [DRAFT]: Added E2E LLM test Added E2E LLM test Oct 21, 2024
akladiev
akladiev reviewed Oct 21, 2024
View reviewed changes
tests/llm/requirements.txt Outdated Show resolved Hide resolved
tests/llm/requirements.txt Show resolved Hide resolved
@AlexKoff88
Copy link
Contributor Author

Can we proceed with the merge? The test suite works on Linux CPU/GPU and WIndows CPU. There is a crash when using GPU on Windows but this is some runtime issue.

@AlexKoff88
Copy link
Contributor Author

Details in Ref. 154161.

@AlexKoff88
Copy link
Contributor Author

I think the PR is ready. Windows issue was fixed on the CI side.

@AlexKoff88
Copy link
Contributor Author

Gentle ping to review and merge.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@ilya-lavrenov ilya-lavrenov ilya-lavrenov left review comments

@akladiev akladiev akladiev left review comments

@akuporos akuporos Awaiting requested review from akuporos akuporos is a code owner automatically assigned from openvinotoolkit/openvino-configuration-mgmt

@artanokhov artanokhov Awaiting requested review from artanokhov artanokhov is a code owner automatically assigned from openvinotoolkit/openvino-configuration-mgmt

At least 1 approving review is required to merge this pull request.

Assignees

@ilya-lavrenov ilya-lavrenov

Labels
category: dependency_changes Pull requests that update a dependency file no-match-files
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@AlexKoff88 @ilya-lavrenov @akladiev