Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[GPU] LSTMSequence and LSTMCell optimization #26767

Open
wants to merge 135 commits into
base: master
Choose a base branch
from
Open

[GPU] LSTMSequence and LSTMCell optimization #26767

wants to merge 135 commits into from
+2,232 −276

Conversation

michal-miotk
Copy link
Contributor

@michal-miotk michal-miotk commented Sep 24, 2024 •
edited
Loading

Details:

  • creating simple primitive for lstm_sequence to be faster than previous approach using many primitives
  • using oneDNN
  • based on commit c99ddc0 from 25732

Tickets:

  • 146601

@michal-miotk
compiles lstm_seq
9ce143a
@michal-miotk
more kernel args
027f991
@michal-miotk
bigger proper run chances
c191c58
@michal-miotk
19jul
d461e66
@michal-miotk
inference works
01fa2ac
@michal-miotk
in middle of implementation
1f017fd
@michal-miotk
problems with inputs get element in kernel
5787c7d
@michal-miotk
not compile
837db22
@michal-miotk
wipx
d4ce531
@michal-miotk
wip
19c268e
@michal-miotk
solved problem with too much inputs kernel
f5273bc
@michal-miotk
wip
d50b3be
@michal-miotk
more changes
63a8dfd
@michal-miotk
wip
f54ecc1
@michal-miotk
wip
3748a11
@michal-miotk
wip
fae772a
@michal-miotk
proper shape for 2 outputs
c00ff8a
@michal-miotk
Squashed commit of the following:
1c08b14 
commit 232d272f11fbe65e82fa9787260a8b9d34b57d20
Author: michal-miotk <michal.miotk@intel.com>
Date:   Mon Jul 29 11:17:47 2024 +0000

    wip

commit e642ca3
Author: michal-miotk <michal.miotk@intel.com>
Date:   Sun Jul 28 22:08:24 2024 +0000

    wip

commit c6b74d3
Author: michal-miotk <michal.miotk@intel.com>
Date:   Fri Jul 26 14:10:26 2024 +0000

    wip

commit 0451429
Author: michal-miotk <michal.miotk@intel.com>
Date:   Thu Jul 25 20:35:11 2024 +0000

    wip3
@michal-miotk
Squashed commit of the following:
6968881 
commit 1164592
Author: michal-miotk <michal.miotk@intel.com>
Date:   Tue Aug 6 09:25:45 2024 +0000

    wip

commit 8b2c049
Author: michal-miotk <michal.miotk@intel.com>
Date:   Tue Aug 6 09:24:02 2024 +0000

    wip

commit 886b412
Author: michal-miotk <michal.miotk@intel.com>
Date:   Mon Aug 5 14:59:14 2024 +0000

    wip

commit 08fb207
Author: michal-miotk <michal.miotk@intel.com>
Date:   Sun Aug 4 20:21:38 2024 +0000

    wip, errors on half

commit 125884d
Author: michal-miotk <michal.miotk@intel.com>
Date:   Sat Aug 3 23:59:58 2024 +0000

    wip

commit af4f209
Author: michal-miotk <michal.miotk@intel.com>
Date:   Fri Aug 2 17:58:38 2024 +0000

    wip

commit 12626fc
Author: michal-miotk <michal.miotk@intel.com>
Date:   Fri Aug 2 10:52:15 2024 +0000

    wip

commit dfdd052
Author: michal-miotk <michal.miotk@intel.com>
Date:   Thu Aug 1 15:38:41 2024 +0000

    wip

commit 54ee912
Author: michal-miotk <michal.miotk@intel.com>
Date:   Thu Aug 1 11:01:55 2024 +0000

    only bfyx layout

commit 240fe4a
Author: michal-miotk <michal.miotk@intel.com>
Date:   Thu Aug 1 10:34:45 2024 +0000

    two outputs from prim

commit bc775be
Author: michal-miotk <michal.miotk@intel.com>
Date:   Wed Jul 31 22:13:14 2024 +0000

    wip

commit d1cfd60
Author: michal-miotk <michal.miotk@intel.com>
Date:   Wed Jul 31 22:07:06 2024 +0000

    wip

commit 7d18884
Author: michal-miotk <michal.miotk@intel.com>
Date:   Wed Jul 31 19:19:04 2024 +0000

    begin of handling reverse

commit 39f64af
Author: michal-miotk <michal.miotk@intel.com>
Date:   Wed Jul 31 15:37:06 2024 +0000

    betterbetter

commit 67b3c9a
Author: michal-miotk <michal.miotk@intel.com>
Date:   Wed Jul 31 13:12:39 2024 +0000

    better

commit 6ded5aa
Author: michal-miotk <michal.miotk@intel.com>
Date:   Wed Jul 31 10:12:31 2024 +0000

    wip

commit 1ccdacc
Author: michal-miotk <michal.miotk@intel.com>
Date:   Tue Jul 30 23:07:21 2024 +0000

    wip

commit ab1307c
Author: michal-miotk <michal.miotk@intel.com>
Date:   Tue Jul 30 22:00:50 2024 +0000

    test passed

commit bc65969
Author: michal-miotk <michal.miotk@intel.com>
Date:   Tue Jul 30 15:37:20 2024 +0000

    wip

commit 03cbf57
Author: michal-miotk <michal.miotk@intel.com>
Date:   Tue Jul 30 15:15:06 2024 +0000

    only 2 outputs

commit fd5f3dc
Author: michal-miotk <michal.miotk@intel.com>
Date:   Tue Jul 30 14:47:12 2024 +0000

    wip

commit 939d23c
Author: michal-miotk <michal.miotk@intel.com>
Date:   Tue Jul 30 11:34:56 2024 +0000

    wip

commit 2bb561f
Author: michal-miotk <michal.miotk@intel.com>
Date:   Tue Jul 30 09:28:03 2024 +0000

    added to binary buffer

commit 1ef83ff
Author: michal-miotk <michal.miotk@intel.com>
Date:   Mon Jul 29 22:30:57 2024 +0000

    not works
@michal-miotk
cleaning
31fcb79
@michal-miotk
Merge branch 'master' into lstm2
4b16eef
@michal-miotk
updated to new primitive_base api, disabled lstm to tensor transforma…
dcad182 
…tion only in gpu plugin
@michal-miotk
now it should compile on windows, changed kernel name
d6aeb54
@michal-miotk
deleted cell, deleted input_forget
9688f63
@michal-miotk
generic primitive
5003d47
@michal-miotk
fix compilation problem, smaller lws
5937b14
@michal-miotk
wip
8b31a91
@michal-miotk
wip, not resolved fail on dynamic
2ff5a7c
@michal-miotk
fixed failing dynamic test
2d9e5c6
@michal-miotk
change name cldnn::rnn -> cldnn::lstm_seq
702e941
Lyamin-Roman
Lyamin-Roman reviewed Oct 18, 2024
View reviewed changes
src/plugins/intel_gpu/include/intel_gpu/graph/program.hpp Outdated Show resolved Hide resolved
src/plugins/intel_gpu/include/intel_gpu/primitives/rnn.hpp Show resolved Hide resolved
src/plugins/intel_gpu/include/intel_gpu/primitives/rnn.hpp Outdated Show resolved Hide resolved
src/plugins/intel_gpu/src/graph/graph_optimizer/post_optimize_lstm_weights_and_output.cpp Outdated Show resolved Hide resolved
src/plugins/intel_gpu/src/graph/impls/ocl/lstm_cell.cpp Outdated Show resolved Hide resolved
src/plugins/intel_gpu/src/graph/lstm_cell.cpp Outdated Show resolved Hide resolved
src/plugins/intel_gpu/src/kernel_selector/cl_kernels/lstm_cell_and_seq_bfyx.cl Outdated Show resolved Hide resolved
src/plugins/intel_gpu/src/kernel_selector/kernels/lstm/lstm_cell_kernel_ref.cpp Outdated Show resolved Hide resolved
src/plugins/intel_gpu/src/plugin/ops/rnn.cpp Outdated Show resolved Hide resolved
src/plugins/intel_gpu/src/plugin/ops/rnn.cpp Outdated Show resolved Hide resolved
@michal-miotk
applying review part1
303bf7d
@michal-miotk
fix check of dimensions
bf9f13f
@michal-miotk
fix check of input dim lstm cell
459e1ad
@michal-miotk
enable onednn for tests ON, LSTMSeq accept bfyx and fbyx format
14e53f4
@michal-miotk
dot op, vec_size=4
063ac02
@michal-miotk
Revert "skipping conv fusion tests"
892131b 
This reverts commit 6af1f3f.
@michal-miotk
Revert "conv_fp32_multi_eltquant skip in conv_fusion_test"
b539a3f 
This reverts commit f8dbec3.
@michal-miotk
lstm_weights optimization is part of post_optimize_weights
dc8ac73
@michal-miotk
fix forbiddnen size_t->int conversion
a5165a8
@michal-miotk
Revert "update hash as input format of weights is custom after post_o…
cc6b4b5 
…ptimize_weights pass"

This reverts commit 00826ad.
@michal-miotk
Merge branch 'master' into lstm_with_onednn
b168fbe
@michal-miotk
inheriting from RNNParams instead of composition
c38c321
@michal-miotk
fix failing tests, added input forget
73cde93
@michal-miotk
Merge branch 'master' into lstm_with_onednn
99a3ca5
@michal-miotk
fix error - not override
b5ca43f
@michal-miotk
reenabling lstm_cell decomposition
1d1deb7
@michal-miotk
no passing R weights reorder to reorder params
f14ed32
@michal-miotk
refactor weights post optimize
fb12ef6
@michal-miotk
fix conversion size_t to int
63bddcb
@michal-miotk
only lstmseq decomposition
2b6ad65
@michal-miotk
little refactor
8e1d36c
@michal-miotk
micro refactor - deleted one line
bd81512
@michal-miotk
Merge branch 'master' into lstm_with_onednn
e4fffa5
@michal-miotk
deleting skip of deconvolution_random_test
aea04bd
@michal-miotk
Revert "only lstmseq decomposition"
7af4e1e 
This reverts commit 2b6ad65.
@michal-miotk
updated onednn
75d0b67
@michal-miotk
oooq only for lstmSeq
acd6369
@michal-miotk
Revert "updated onednn"
0430ee3 
This reverts commit 75d0b67.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@vladimir-paramuzov vladimir-paramuzov vladimir-paramuzov left review comments

@Lyamin-Roman Lyamin-Roman Lyamin-Roman left review comments

At least 1 approving review is required to merge this pull request.

Assignees
No one assigned
Labels
category: build OpenVINO cmake script / infra category: GPU OpenVINO GPU plugin category: IE Tests OpenVINO Test: plugins and common
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@michal-miotk @vladimir-paramuzov @Lyamin-Roman