Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

EliminateAddZero #29269

Open
wants to merge 1 commit into
base: master
Choose a base branch
from
Open

EliminateAddZero #29269

wants to merge 1 commit into from
+83 −2

Conversation

wine99
Copy link
Contributor

@wine99 wine99 commented Mar 4, 2025 •
edited
Loading

Changes:

  • Add EliminateAddZero in LinOpSequenceFusion
  • Add another run of LinOpSequenceFusion in CPU transformations pipeline after ConvertPrecision

Purpose:

Some LLM models generated by onnxruntime-genai has this pattern:
image
The result of Sub will be converted to i32 and fed into GroupQueryAttention. This PR #28163 decomposes GQA and introduces an Add that adds 1 back to the result.

With the changes in this PR, this add1 sub1 pattern which is repeated multiple times could be completely eliminated.

@wine99 wine99 requested review from a team as code owners March 4, 2025 08:44
@wine99 wine99 requested review from itikhono and removed request for a team March 4, 2025 08:44
@github-actions github-actions bot added category: CPU OpenVINO CPU plugin category: transformations OpenVINO Runtime library - Transformations labels Mar 4, 2025
@sys-openvino-ci sys-openvino-ci added the ExternalPR External contributor label Mar 4, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@itikhono itikhono Awaiting requested review from itikhono itikhono is a code owner automatically assigned from openvinotoolkit/openvino-ie-transformations-maintainers

At least 1 approving review is required to merge this pull request.

Assignees
No one assigned
Labels
category: CPU OpenVINO CPU plugin category: transformations OpenVINO Runtime library - Transformations ExternalPR External contributor
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@wine99 @sys-openvino-ci