Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fine tune MUL_MAT, new threading (spin+wait/notify), speedup q_f32 BLAS by splitting COMPUTE stage #1632

Closed
wants to merge 24 commits into from
Closed

Fine tune MUL_MAT, new threading (spin+wait/notify), speedup q_f32 BLAS by splitting COMPUTE stage #1632

wants to merge 24 commits into from

Commits on Jun 18, 2023

  1. initial

    @mqy
    mqy committed Jun 18, 2023
    Configuration menu
    Copy the full SHA
    213f133 View commit details
    Browse the repository at this point in the history

  2. threading test: improve readability at both codes and output

    @mqy
    mqy committed Jun 18, 2023
    Configuration menu
    Copy the full SHA
    1b041d7 View commit details
    Browse the repository at this point in the history

  3. bulk refactored task profile to support complete fallback; enable tun… 

    …e by default for ease of dev
    @mqy
    mqy committed Jun 18, 2023
    Configuration menu
    Copy the full SHA
    48016f6 View commit details
    Browse the repository at this point in the history

  4. threading test: At github, Windows can take more than 20 seconds to s… 

    …tart 15 threads.Let's silently ignore when we saw two adjacent slowness.
    @mqy
    mqy committed Jun 18, 2023
    Configuration menu
    Copy the full SHA
    9106232 View commit details
    Browse the repository at this point in the history

  5. Workrounnd to set node->backend

    @mqy
    mqy committed Jun 18, 2023
    Configuration menu
    Copy the full SHA
    bb590f1 View commit details
    Browse the repository at this point in the history

  6. tunning: check GPU offloading before loading model

    @mqy
    mqy committed Jun 18, 2023
    Configuration menu
    Copy the full SHA
    7c05049 View commit details
    Browse the repository at this point in the history

  7. tunning: add f16, todo: f32 failed with CL

    @mqy
    mqy committed Jun 18, 2023
    Configuration menu
    Copy the full SHA
    21e9379 View commit details
    Browse the repository at this point in the history

  8. tunning: support k_quants; disabled rope shapes (workaround); make ca… 

    …che thread safe; fixed shape comprison
    @mqy
    mqy committed Jun 18, 2023
    Configuration menu
    Copy the full SHA
    5342dc0 View commit details
    Browse the repository at this point in the history

  9. try make CL run w/o tunning, but -ngl stucks no output. had to add ta… 

    …sk runer and profile id, many changes, see the f codes
    @mqy
    mqy committed Jun 18, 2023
    Configuration menu
    Copy the full SHA
    6b83a3e View commit details
    Browse the repository at this point in the history

  10. bulk refactoring task profile and related to run CL GPU offloading. 

    * removed ggml_task_backend, infavour of ggml_task_profile.runner and newly added id and name.
* extracted mul_mat blas codes into ggml_compute_forward_mul_mat_blas,
  thus align with CUDA/CL a bit more and make it easier to fix profile and run tune.
* rewrote task profile and update/add some cuda/cl codes, finnaly made CL GPU offloading work.
* misc minor fix/update to tune, the data format was changed.
    @mqy
    mqy committed Jun 18, 2023
    Configuration menu
    Copy the full SHA
    06b0082 View commit details
    Browse the repository at this point in the history

  11. typos

    @mqy
    mqy committed Jun 18, 2023
    Configuration menu
    Copy the full SHA
    67bb367 View commit details
    Browse the repository at this point in the history

  12. fix cuda build error

    @mqy
    mqy committed Jun 18, 2023
    Configuration menu
    Copy the full SHA
    2193ab6 View commit details
    Browse the repository at this point in the history

  13. fixed break and asssion from select; try fix cuda link error

    @mqy
    mqy committed Jun 18, 2023
    Configuration menu
    Copy the full SHA
    0ec4dab View commit details
    Browse the repository at this point in the history

  14. fix warning

    @mqy
    mqy committed Jun 18, 2023
    Configuration menu
    Copy the full SHA
    5abb8ae View commit details
    Browse the repository at this point in the history

  15. threading: add suspend/resume APIs, so it's possible to run a thread … 

    …pool at session level
    @mqy
    mqy committed Jun 18, 2023
    Configuration menu
    Copy the full SHA
    5feefb3 View commit details
    Browse the repository at this point in the history

  16. threadng: remove unnecessary spin lock/unlock from suspend/resume; ad… 

    …d more tests
    @mqy
    mqy committed Jun 18, 2023
    Configuration menu
    Copy the full SHA
    286c5b3 View commit details
    Browse the repository at this point in the history

  17. threading test: less loops to avoid timeout

    @mqy
    mqy committed Jun 18, 2023
    Configuration menu
    Copy the full SHA
    9872863 View commit details
    Browse the repository at this point in the history

  18. fixed OP_OUT_PROD and OP_NONE

    @mqy
    mqy committed Jun 18, 2023
    Configuration menu
    Copy the full SHA
    6609c22 View commit details
    Browse the repository at this point in the history

Commits on Jun 19, 2023

  1. tune: update readme

    @mqy
    mqy committed Jun 19, 2023
    Configuration menu
    Copy the full SHA
    65fd65e View commit details
    Browse the repository at this point in the history

  2. tune: extract ggml_mulmat_tune_bench_wrapper

    @mqy
    mqy committed Jun 19, 2023
    Configuration menu
    Copy the full SHA
    44b831d View commit details
    Browse the repository at this point in the history

  3. threading test: decrease a threshold value to avoid timeout

    @mqy
    mqy committed Jun 19, 2023
    Configuration menu
    Copy the full SHA
    4d32b40 View commit details
    Browse the repository at this point in the history

  4. threading: fix deadlock by reverting part of changes from commit 286c5b3

    @mqy
    mqy committed Jun 19, 2023
    Configuration menu
    Copy the full SHA
    cc8a375 View commit details
    Browse the repository at this point in the history

  5. threading: try to fix a deadlock, also added critical deadlock detection

    @mqy
    mqy committed Jun 19, 2023
    Configuration menu
    Copy the full SHA
    aac7f7c View commit details
    Browse the repository at this point in the history

  6. threading: removed feature wait_on_done to figure out causes of deadl… 

    …ock in windows AVX
    @mqy
    mqy committed Jun 19, 2023
    Configuration menu
    Copy the full SHA
    08972d2 View commit details
    Browse the repository at this point in the history