Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Merge branch 'submission-v4.1' into 'master' #949

Draft
wants to merge 21 commits into
base: master
Choose a base branch
from
Draft

Merge branch 'submission-v4.1' into 'master' #949

wants to merge 21 commits into from

Conversation

anhappdev
Copy link
Collaborator

@anhappdev anhappdev commented Jan 15, 2025 •
edited
Loading

The branch anh/merge-v4.1-into-master is used to resolve merge conflicts when merging submission-v4.1 into master.

anhappdev and others added 19 commits September 6, 2024 17:06
@anhappdev
Enable stable_diffusion tests
513dcf1
@anhappdev
Merge branch 'refs/heads/master' into submission-v4.1
d92f069
@anhappdev
Add a caption_id to coco_gen dataset (#918)
723d323
@mohitmundhragithub
Final Submission for code for Qualcomm
eea4595 
co-author: Aswin B <quic_aswib@quicinc.com>
co-author: Utkarsh Mishra <quic_utkarshm@quicinc.com>
@mohitmundhragithub
Addressing review comments
bcf4215
@anhappdev
Update seed and num_steps for TFLite SD task (#16)
0ae020e
@mohitmundhragithub
Ran make format
acb17bf
@mohitmundhragithub
Applying linter changes
b6ee8e7
@freedomtan
Merge pull request #9 from mlcommons/submission_v4.1_qti
c811734 
Final Submission for code for Qualcomm
@anhappdev
Update QTI backend for submission v4.1 (#13)
786a485 
* Ignore QTI private SDK and libs

* Update path for QTI SDK

* Download QTI libs for SD task

* Fix QTI BUILD file

* Update integration test

* Set WITH_STABLEDIFFUSION=1 for CI build

* Fix QTI pbtxt files

* Disable SD task for Apple devices

* Update model path and checksum for QTI

* Remove tiny groundtruth_path for SD task

* Update path for TFLite models

* Update groundtruth_path for SD task
@anhappdev
Update tflite_settings_mtk_mt6989.pbtxt
fad6765
@freedomtan
enable stable diffusion in Pixel backend (#936)
f74b27c 
* hacks to enable stable diffusion on Pixel backend

* buildifer and clang-format

* remove unnecessary line

* adapt to submission-v4.1
@anhappdev
feat: add icon and description for Stable Diffusion benchmark (#917)
aab2697 
* Add icon for stable_diffusion task

* Add description for stable_diffusion task

* Sort the order of task based on BenchmarkId.allIds

* Fix ios-build-test.yml

* Fix ios-build-test.yml
@RSMNYS @anhappdev
use time step embedding from file (#928)
48654bd 
* feat: pass task-specific config to backend (#922)

* Add TaskConfig.CustomConfig and pass them to backend

* Add CustomConfig for main.cc

* Use seed and num_steps from CustomConfig for TFLite backend

* Replace std::cout with LOG(INFO)

* Format files

* feat: add ConvertOutputs() API (#927)

* Add ConvertOutputs() API

* Add ConvertOutputs() for mobile_back_tflite

* Set minimum macos version

* Set minimum macos version to 13.1

* Update _kIphoneOnGitHubAction

* feat: timestamp-embedding-parser (WIP)

* disabled bitcode to be able compile with new XCode

* chore: formatting

* refactor: use custom setting in Core ML backend to detect NCHW input. (#924)

* Add GetConfigValue()

* Add custom setting data-format for Core ML

* Use GetConfigValue() to get stable_diffusion_seed and stable_diffusion_num_steps

* fix: resolve crash due to permission denied on Android Play Store version (#930)

* Set android:extractNativeLibs="true"

* Set android.bundle.enableUncompressedNativeLibs=false

* chore: increase Android minSdkVersion from 21 to 30 (#859)

Increase minSdkVersion to 30

* feat: finalized SD pipeline to use embedding from the binary file.

* refactor: updated embedding_utils to parse pkl file

* chore: linting

* fix: fixed lint issue in neuron

* chore: BUILD cleanup

* chore: cleanup

* chore: ignore .fvm

* chore: updated model paths and checksums for stable diffusion benchmark: tflite_settings_android.pbtxt

* chore: reverse timesteps and embeddings to support descending order of the timesteps and embeddings

* chore: fixed formatting

* chore: added links to the sd models and timestep embeddings file

* chore: add the proper name for the embedding_timesteps file

* chore: added missed declaration for backend_convert_outputs

* chore: clang formatting

* chore: added missed files

* chore: fixed build file for the pixel backend

* chore: bazel formatting

* fix: added missed interface implementation for pixel

* chore: clang formatting

---------

Co-authored-by: Anh <anh.app.dev@gmail.com>
@anhappdev
Update model_path for stable_diffusion (#946)
f7fc2e8
@anhappdev
Merge branch 'submission-v4.1' into anh/merge-v4.1-into-master
30483df 
# Conflicts:
#	flutter/assets/tasks.pbtxt
#	flutter/cpp/datasets/coco_gen.cc
#	flutter/cpp/proto/mlperf_task.proto
#	flutter/integration_test/expected_accuracy.dart
#	flutter/integration_test/expected_throughput.dart
#	mobile_back_apple/dev-utils/Makefile
#	mobile_back_qti/.gitignore
#	mobile_back_qti/cpp/backend_qti/settings/qti_settings_default_cpu.pbtxt
#	mobile_back_qti/cpp/backend_qti/settings/qti_settings_default_dsp.pbtxt
#	mobile_back_qti/cpp/backend_qti/settings/qti_settings_default_gpu.pbtxt
#	mobile_back_qti/cpp/backend_qti/settings/qti_settings_gpufp16.pbtxt
#	mobile_back_qti/cpp/backend_qti/settings/qti_settings_sd7cxg3.pbtxt
#	mobile_back_qti/cpp/backend_qti/settings/qti_settings_sd7g1.pbtxt
#	mobile_back_qti/cpp/backend_qti/settings/qti_settings_sd7pg2.pbtxt
#	mobile_back_qti/cpp/backend_qti/settings/qti_settings_sd8cxg3.pbtxt
#	mobile_back_qti/cpp/backend_qti/settings/qti_settings_sd8g1.pbtxt
#	mobile_back_qti/cpp/backend_qti/settings/qti_settings_sd8g2.pbtxt
#	mobile_back_qti/cpp/backend_qti/settings/qti_settings_sd8g3.pbtxt
#	mobile_back_qti/cpp/backend_qti/settings/qti_settings_sd8pg1.pbtxt
#	mobile_back_qti/cpp/backend_qti/settings/qti_settings_sdm778.pbtxt
#	mobile_back_qti/cpp/backend_qti/settings/qti_settings_sdm888.pbtxt
#	mobile_back_qti/cpp/backend_qti/settings/qti_settings_sm4450.pbtxt
#	mobile_back_qti/cpp/backend_qti/settings/qti_settings_sm7550.pbtxt
#	mobile_back_qti/cpp/backend_qti/settings/qti_settings_sm8635.pbtxt
#	mobile_back_tflite/cpp/backend_tflite/neuron/backend_settings/tflite_settings_mtk_mt6989.pbtxt
#	mobile_back_tflite/cpp/backend_tflite/stable_diffusion_invoker.cc
@anhappdev
Use CloudFare link for qti model_path
46c839a
@anhappdev
Use CloudFare link for tflite model_path
6648c56
@anhappdev
Fix build error
dcd283f
@github-actions GitHub Actions
Copy link

github-actions bot commented Jan 15, 2025 •
edited
Loading

MLCommons CLA bot All contributors have signed the MLCommons CLA ✍️ ✅

@anhappdev
Copy link
Collaborator Author

@mohitmundhragithub It looks like the QTI backend blocks the fallback to other backends.

01-15 12:43:51.577 I/native  (24059): cpp/flutter/dart_backend_match.cc:13 checking backend 'libqtibackend.so' ...
01-15 12:43:51.586 I/native  (24059): cpp/backends/external.cc:195 Using backend allocator
01-15 12:43:51.586 I/native  (24059): cpp/backend_qti/qti_c.cc:69 QTI HW supported check: model: SM-M326B, manufacturer: samsung
01-15 12:43:51.586 I/native  (24059): cpp/backend_qti/qti_c.cc:52 adsp_lib_path: /data/app/~~9QKpM5zSKB1qXMWAegy5pg==/org.mlcommons.android.mlperfbench-EHDggiH2wNGyLB4x6AoIjw==/lib/arm64;/system/lib/rfsa/adsp;/system/vendor/lib/rfsa/adsp;/dsp
01-15 12:43:51.586 I/native  (24059): cpp/backend_qti/qti_c.cc:57 ld_lib_path: /data/app/~~9QKpM5zSKB1qXMWAegy5pg==/org.mlcommons.android.mlperfbench-EHDggiH2wNGyLB4x6AoIjw==/lib/arm64;/system/vendor/lib64
01-15 12:43:51.586 I/native  (24059): cpp/backend_qti/soc_utility.cc:139 Failed to read SOC file: 
01-15 12:43:51.586 I/native  (24059): cpp/backend_qti/soc_utility.cc:317 Soc ID: 0
01-15 12:43:51.589 F/native  (24059): cpp/backend_qti/qti_backend_helper.cc:124 runtime 0 is not available on this platform

log-Samsung Galaxy M32.txt
log-Samsung Galaxy S24.txt

For reference, here is the order of check:

const _backendsList = [
  'MEDIATEK_TAG',
  'PIXEL_TAG',
  'QTI_TAG',
  'SAMSUNG_TAG',
  'APPLE_TAG',
  'TFLITE_TAG',
];

@freedomtan
Copy link
Contributor

freedomtan commented Jan 15, 2025 •
edited
Loading

@anhappdev let's create 5.0 submission branch from the 4.1 branch if we cannot do it from the master branch. The submission date is next Fri. @mohitmundhragithub and @Mostelk. For submission, we don't need an integrated one working.

@anhappdev
Copy link
Collaborator Author

@anhappdev let's create 5.0 submission branch from the 4.1 branch if we cannot do it from the master branch. The submission date is next Fri. @mohitmundhragithub and @Mostelk. For submission, we don't need an integrated one working.

Should we create the 5.0 branch from this PR / branch? Since it contains all the commits from both master and submission-v4.1. Then we will have all the latest changes, and also avoid a lot of merge conflicts later.

@sonarqubecloud SonarQubeCloud
Copy link

Quality Gate Passed Quality Gate passed

Issues
23 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

@freedomtan
Copy link
Contributor

@anhappdev let's create 5.0 submission branch from the 4.1 branch if we cannot do it from the master branch. The submission date is next Fri. @mohitmundhragithub and @Mostelk. For submission, we don't need an integrated one working.

Should we create the 5.0 branch from this PR / branch? Since it contains all the commits from both master and submission-v4.1. Then we will have all the latest changes, and also avoid a lot of merge conflicts later.

sounds good to me.

@mohitmundhragithub and @Mostelk

@anhappdev
Copy link
Collaborator Author

https://github.com/mlcommons/mobile_app_open/tree/submission-v5.0

@mohitmundhragithub
Copy link
Contributor

Hi @anhappdev, Can you please upload the ts-embedding pkl file as well, preferably in mobile_open?
Currently only the .bin.ts file is present.

@anhappdev
Copy link
Collaborator Author

Hi @anhappdev, Can you please upload the ts-embedding pkl file as well, preferably in mobile_open? Currently only the .bin.ts file is present.

Ok. I uploaded it here: https://github.com/mlcommons/mobile_open/releases/tag/v4.1-sd-tflite

@mohitmundhragithub
Copy link
Contributor

Hi @anhappdev, Can you please upload the ts-embedding pkl file as well, preferably in mobile_open? Currently only the .bin.ts file is present.

Ok. I uploaded it here: https://github.com/mlcommons/mobile_open/releases/tag/v4.1-sd-tflite

#928 (comment)

the pkl file you uploaded is different from the one that is generated in this comment.
Can you please update the pkl file?

@anhappdev
Copy link
Collaborator Author

the pkl file you uploaded is different from the one that is generated in this comment. Can you please update the pkl file?

Ok. I replaced the file with the new one. Please check again.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@anhappdev @freedomtan @mohitmundhragithub @RSMNYS