Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[NPU] Add plugin metadata to compiled model #27159

Merged
merged 23 commits into from
Jan 15, 2025
Merged

[NPU] Add plugin metadata to compiled model #27159

merged 23 commits into from
Jan 15, 2025

Conversation

alexandruenache1111
Copy link
Contributor

@alexandruenache1111 alexandruenache1111 commented Oct 21, 2024 •
edited
Loading

Details:

A versioned blob would look like this:
blob-layout

Where Blob data size is necessary when importing the blob to offset jump at Metadata.

Rules of thumb when we need to set a new Metadata version:

  • if we add fields at the end of current Metadata, then Minor is incremented. This means we maintain backward compatibility with old blobs.

Example:
After OV Version we add a new field.

  • if we remove fields or add fields anywhere but at end, then Major is incremented. This means we break forward compatibility with old blobs.

Examples:
Between Minor and OV Version, we add a new field.
OR
We remove OV Version.

Simplified workflow:

  • at export: we append the versioning stuff at the end of the blob
  • at import: we extract the stored Metadata and verify it against the currently supported one. If the imported blob is incompatible, we reject it and print its OV version (where possible).

Tickets:

  • E-135371

@github-actions github-actions bot added the category: NPU OpenVINO NPU plugin label Oct 21, 2024
@alexandruenache1111 alexandruenache1111 self-assigned this Oct 25, 2024
@alexandruenache1111 alexandruenache1111 marked this pull request as ready for review October 25, 2024 09:50
@alexandruenache1111 alexandruenache1111 requested review from a team as code owners October 25, 2024 09:50
@alexandruenache1111 alexandruenache1111 marked this pull request as draft November 11, 2024 17:12
@alexandruenache1111 alexandruenache1111 force-pushed the blob_stream_commit branch 2 times, most recently from aaba4a6 to 421d071 Compare November 12, 2024 06:45
@alexandruenache1111 alexandruenache1111 marked this pull request as ready for review November 12, 2024 06:46
@github-actions github-actions bot added the category: build OpenVINO cmake script / infra label Nov 28, 2024
StefaniaHergane
StefaniaHergane reviewed Nov 28, 2024
View reviewed changes
Copy link
Contributor

@StefaniaHergane StefaniaHergane left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Small renaming suggestion for createMetadata().

src/plugins/intel_npu/src/plugin/src/metadata.cpp Outdated Show resolved Hide resolved
@sbutnari
Copy link
Contributor

build_jenkins

razvanapetroaie
razvanapetroaie reviewed Dec 5, 2024
View reviewed changes
src/plugins/intel_npu/src/plugin/include/metadata.hpp Outdated Show resolved Hide resolved
src/plugins/intel_npu/src/plugin/src/metadata.cpp Outdated Show resolved Hide resolved
src/plugins/intel_npu/src/plugin/include/metadata.hpp Outdated Show resolved Hide resolved
src/plugins/intel_npu/src/plugin/src/metadata.cpp Outdated Show resolved Hide resolved
src/plugins/intel_npu/src/plugin/src/metadata.cpp Outdated Show resolved Hide resolved
src/plugins/intel_npu/src/plugin/src/metadata.cpp Outdated Show resolved Hide resolved
src/plugins/intel_npu/src/plugin/src/metadata.cpp Outdated Show resolved Hide resolved
src/plugins/intel_npu/src/plugin/include/metadata.hpp Outdated Show resolved Hide resolved
src/plugins/intel_npu/src/plugin/src/metadata.cpp Outdated Show resolved Hide resolved
src/plugins/intel_npu/tests/unit/CMakeLists.txt Show resolved Hide resolved
@alexandruenache1111 alexandruenache1111 requested review from a team as code owners December 6, 2024 08:10
@github-actions github-actions bot added category: tools OpenVINO C++ / Python tools category: OVC OVC tool labels Dec 6, 2024
@alexandruenache1111 alexandruenache1111 removed the request for review from a team December 6, 2024 08:12
@alexandruenache1111
Added test for reading any invalid metadata version
21937de 
Signed-off-by: Alexandru Enache <alexandru.enache@intel.com>
@alexandruenache1111
Change some test names
bee4350 
Signed-off-by: Alexandru Enache <alexandru.enache@intel.com>
@alexandruenache1111
Updated writeAndReadInvalidMetadataVersion test body
3daf98a 
Signed-off-by: Alexandru Enache <alexandru.enache@intel.com>
@alexandruenache1111
Updated comment regarding metadata version field rule
d4a29bd 
Signed-off-by: Alexandru Enache <alexandru.enache@intel.com>
@alexandruenache1111
Clang format + small retouches
e6e0ee3 
Signed-off-by: Alexandru Enache <alexandru.enache@intel.com>
@alexandruenache1111
Move version functions as static methods inside MetadataBase
15c56bb 
Signed-off-by: Alexandru Enache <alexandru.enache@intel.com>
@alexandruenache1111
Nitpicks
7f7e4a8 
Signed-off-by: Alexandru Enache <alexandru.enache@intel.com>
@alexandruenache1111
Moved the version mismatch print to throw call
4e959c3 
Signed-off-by: Alexandru Enache <alexandru.enache@intel.com>
@alexandruenache1111
Handling case with newer minor version + improved catch statement ins…
d7225ff 
…ide read_metadata_from

Signed-off-by: Alexandru Enache <alexandru.enache@intel.com>
@alexandruenache1111
Fix test fixture logic + added separate test for newer minor version …
14f5018 
…case

Signed-off-by: Alexandru Enache <alexandru.enache@intel.com>
@alexandruenache1111
Clang format
6d304b1 
Signed-off-by: Alexandru Enache <alexandru.enache@intel.com>
@alexandruenache1111
Remove unused test vars
1bbf5fd 
Signed-off-by: Alexandru Enache <alexandru.enache@intel.com>
@alexandruenache1111
Add log message for when blob compatibility is skipped by env var
b4f887f 
Signed-off-by: Alexandru Enache <alexandru.enache@intel.com>
@alexandruenache1111
Added tests for newer/removed fields cases
113b17e 
Signed-off-by: Alexandru Enache <alexandru.enache@intel.com>
@alexandruenache1111
Merge branch 'master' into blob_stream_commit
7e3bb33
@alexandruenache1111
Move version field to MetadataBase
1217378 
Signed-off-by: Alexandru Enache <alexandru.enache@intel.com>
@PatrikStepan PatrikStepan added this to the 2025.0 milestone Jan 13, 2025
@moslex moslex added the priority: high High piority label Jan 15, 2025
@alexandruenache1111
Update copyright date
c5e6ec3 
Signed-off-by: Alexandru Enache <alexandru.enache@intel.com>
@PatrikStepan PatrikStepan added this pull request to the merge queue Jan 15, 2025
Merged via the queue into openvinotoolkit:master with commit 6e489ae Jan 15, 2025
161 checks passed
MirceaDan99 pushed a commit to MirceaDan99/openvino that referenced this pull request Jan 22, 2025
@alexandruenache1111 @MirceaDan99
[NPU] Add plugin metadata to compiled model (openvinotoolkit#27159)
1fcd3c0 
### Details:
A versioned blob would look like this:

![blob-layout](https://github.com/user-attachments/assets/4f7f9427-503c-4ee7-bd2c-d40fc2b4df3f)



Where _Blob data size_ is necessary when importing the blob to offset
jump at _Metadata_.

Rules of thumb when we need to set a new `Metadata` version:
- if we **add fields at the end** of current `Metadata`, then `Minor` is
incremented. This means we maintain backward compatibility with old
blobs.

Example:
`After OV Version we add a new field.`
- if we **remove fields** or **add fields anywhere but at end**, then
`Major` is incremented. This means we break forward compatibility with
old blobs.

Examples:
`Between Minor and OV Version, we add a new field.`
OR
`We remove OV Version.`

Simplified workflow:
- at export: we append the versioning stuff at the end of the blob
- at import: we extract the stored `Metadata` and verify it against the
currently supported one. If the imported blob is incompatible, we reject
it and print its OV version (where possible).

### Tickets:
 - *E-135371*

---------

Signed-off-by: Alexandru Enache <alexandru.enache@intel.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@XinWangIntel XinWangIntel XinWangIntel left review comments

@sbutnari sbutnari sbutnari left review comments

@PatrikStepan PatrikStepan PatrikStepan left review comments

@pereanub pereanub pereanub left review comments

@razvanapetroaie razvanapetroaie razvanapetroaie approved these changes

@StefaniaHergane StefaniaHergane StefaniaHergane approved these changes

@lmielick lmielick Awaiting requested review from lmielick

Assignees

@pereanub pereanub

@razvanapetroaie razvanapetroaie

@alexandruenache1111 alexandruenache1111

Labels
category: build OpenVINO cmake script / infra category: NPU OpenVINO NPU plugin Code Freeze priority: high High piority
Projects
None yet
Milestone
2025.0
Development

Successfully merging this pull request may close these issues.
9 participants
@alexandruenache1111 @sbutnari @pereanub @PatrikStepan @XinWangIntel @lmielick @StefaniaHergane @razvanapetroaie @moslex