Code

server: init functional tests#5566

Code

Merged

phymbert merged 100 commits intoggerganov:masterggerganov/llama.cpp:masterfrom phymbert:test/server-add-ci-testphymbert/llama.cpp:test/server-add-ci-testCopy head branch name to clipboardFeb 24, 2024

+1,243-18

Commits

Commits on Feb 20, 2024

• server: init functional test

  phymbert

  committed
  157bcf2

  Copy full SHA for 157bcf2

• server: tests: reduce number of files, all in one tests shell script

  phymbert

  committed
  9b63d70

  Copy full SHA for 9b63d70

• server: tests: fix ci workflow

  phymbert

  committed
  6497755

  Copy full SHA for 6497755

• server: tests: fix ci workflow

  phymbert

  committed
  4e5245e

  Copy full SHA for 4e5245e

• server: tests: fix ci workflow

  phymbert

  committed
  30aa323

  Copy full SHA for 30aa323

• server: tests: use ngxson llama_xs_q4.bin

  phymbert

  committed
  fe9866a

  Copy full SHA for fe9866a

• server: tests: build only the server

  phymbert

  committed
  1680599

  Copy full SHA for 1680599

• server: tests: add health check and concurrent request example

  phymbert

  committed
  8bb586b

  Copy full SHA for 8bb586b

• server: tests: change model to: @karpathy's tinyllamas

  phymbert

  committed
  6c95ec6

  Copy full SHA for 6c95ec6

• server: tests: refactor steps and vocabulary

  phymbert

  committed
  56583be

  Copy full SHA for 56583be

• server: tests: add OAI stream test, fix file end of line, fast fail behave

  phymbert

  committed
  9b7ea97

  Copy full SHA for 9b7ea97

• server: tests: add OAI multi user scenario

  phymbert

  committed
  11adf1d

  Copy full SHA for 11adf1d

• server: tests: slots endpoint checks

  phymbert

  committed
  c355f76

  Copy full SHA for c355f76

• server: tests: check for infinite loops

  phymbert

  committed
  367b59a

  Copy full SHA for 367b59a

• server: tests: check for infinite loops

  phymbert

  committed
  b9f8390

  Copy full SHA for b9f8390

• server: tests: add a constant seed in completion request

  phymbert

  committed
  0772884

  Copy full SHA for 0772884

• server: tests: add infinite loop

  phymbert

  committed
  6b9dc4f

  Copy full SHA for 6b9dc4f

• server: tests: add infinite loop scenario

  phymbert

  committed
  68574c6

  Copy full SHA for 68574c6

• server: tests: add infinite loop scenario

  phymbert

  committed
  b0b6d83

  Copy full SHA for b0b6d83

• server: tests: disable issue 3969 scenario

  phymbert

  committed
  1ecda0d

  Copy full SHA for 1ecda0d

• server: tests: add embeddings scenario

  phymbert

  committed
  e6d4820

  Copy full SHA for e6d4820

• server: tests: add tokenize/detokenize scenario

  phymbert

  committed
  1065f6d

  Copy full SHA for 1065f6d

• server: tests: detokenize endpoint issue reference added

  phymbert

  committed
  19664b9

  Copy full SHA for 19664b9

• server: tests: simplify completion scenario

  phymbert

  committed
  6dcbcfe

  Copy full SHA for 6dcbcfe

Commits on Feb 21, 2024

• server: tests: CORS and api key checks scenario

  phymbert

  committed
  672d98f

  Copy full SHA for 672d98f

• server: tests: add a small check to be sure all started threads have generated response

  phymbert

  committed
  3322bfa

  Copy full SHA for 3322bfa

• server: tests: change CI workflow trigger

  phymbert

  committed
  469af4b

  Copy full SHA for 469af4b

• server: tests: fix the multi users infinite loop test

  phymbert

  committed
  2a37bd6

  Copy full SHA for 2a37bd6

• server : fix initialization thread issues

  ggerganov

  committed
  f1d4138

  Copy full SHA for f1d4138

• server: test: ci change the GitHub workflow trigger

  phymbert

  committed
  600cbeb

  Copy full SHA for 600cbeb

• Merge remote-tracking branch 'origin/master' into test/server-add-ci-test

  phymbert

  committed
  68b8d4e

  Copy full SHA for 68b8d4e

• server: tests:

  Show description for 6406208
  phymbert

  committed
  6406208

  Copy full SHA for 6406208

• server: tests: ci fix model download path

  phymbert

  committed
  01cca66

  Copy full SHA for 01cca66

• server: tests: ci tests.sh exit code

  phymbert

  committed
  534998d

  Copy full SHA for 534998d

Commits on Feb 22, 2024

• minor : fix missing new line

  ggerganov

  committed
  a697cd1

  Copy full SHA for a697cd1

• ci : actually no reason to exclude GPU code from triggers

  ggerganov

  committed
  41676d9

  Copy full SHA for 41676d9

• server: fix health/slots endpoint slot state access available race condition

  phymbert

  committed
  016b221

  Copy full SHA for 016b221

• server: tests: switch to asyncio for concurrent tests, match result content with regex

  phymbert

  committed
  e43406e

  Copy full SHA for e43406e

• server: tests: ci do not take a model anymore, fix trigger patch

  phymbert

  committed
  597c181

  Copy full SHA for 597c181

• Merge remote-tracking branch 'origin/master' into test/server-add-ci-test

  phymbert

  committed
  8b96bda

  Copy full SHA for 8b96bda

• server: tests: ci ensure the server is stopped before scenario, and do not quit while the server is listening

  phymbert

  committed
  f820e10

  Copy full SHA for f820e10

• server: tests: add Multi users with total number of tokens to predict exceeds the KV Cache size

  phymbert

  committed
  aa591ef

  Copy full SHA for aa591ef

• server: tests: Fix some random behavior where the wait for busy status is missing

  phymbert

  committed
  26b66c5

  Copy full SHA for 26b66c5

• server: tests: ci triggered on any changes on server example path

  phymbert

  committed
  51f5274

  Copy full SHA for 51f5274

• server: tests: minor fix missing param.

  phymbert

  committed
  cba6d4e

  Copy full SHA for cba6d4e

Commits on Feb 23, 2024

• server: tests: assert embeddings are actually computed, make the embeddings endpoint configurable.

  Show description for 1bd07e5
  phymbert

  committed
  1bd07e5

  Copy full SHA for 1bd07e5

• server: tests: minor color change

  phymbert

  committed
  14b6ede

  Copy full SHA for 14b6ede

• server: tests: minor fix server --alias param passed twice

  phymbert

  committed
  b38b9e6

  Copy full SHA for b38b9e6

• server: tests: add log in server start to identify why the server does not listen on the CI

  phymbert

  committed
  70e9055

  Copy full SHA for 70e9055

• server: tests: allow to override the server port before launching tests

  phymbert

  committed
  2f756f8

  Copy full SHA for 2f756f8

• server: tests: ci adding container to specify server port and allow the server to listen to

  phymbert

  committed
  6a215e5

  Copy full SHA for 6a215e5

• server: tests: ci adding cmake as it is not present by default in ubuntu base

  phymbert

  committed
  2bb4732

  Copy full SHA for 2bb4732

• server: tests: ci adding python3-pip as it is not present by default in ubuntu base

  phymbert

  committed
  d0e0050

  Copy full SHA for d0e0050

• server: tests: ci adding curl as it is not present by default in ubuntu base for the hf.sh script

  phymbert

  committed
  6e71126

  Copy full SHA for 6e71126

• server: tests: ci adding psmisc as it is not present by default in ubuntu base killall

  phymbert

  committed
  6bba3be

  Copy full SHA for 6bba3be

• server: tests: fix coloring console

  phymbert

  committed
  5110de0

  Copy full SHA for 5110de0

• server: tests: reducing n_ctx and n_predict for // prompts as it is too slow in the CI.

  phymbert

  committed
  bedf37c

  Copy full SHA for bedf37c

• server: tests: reducing sleep time during scenario

  phymbert

  committed
  530d3ae

  Copy full SHA for 530d3ae

• server: tests: parallel fix server is started twice, add colors to help to monitor in the CI jobs

  phymbert

  committed
  36ddb96

  Copy full SHA for 36ddb96

• server: tests: ci : build and run tests for all matrix defines, sanitizer and type

  phymbert

  committed
  0b0f056

  Copy full SHA for 0b0f056

• server: tests: ci : fix wget missing

  phymbert

  committed
  29f8833

  Copy full SHA for 29f8833

• server: tests: ci : add git

  phymbert

  committed
  12bb797

  Copy full SHA for 12bb797

• server: tests: ci : matrix cuda

  phymbert

  committed
  68cd1a4

  Copy full SHA for 68cd1a4

• server: tests: ci : continue on error

  phymbert

  committed
  86896aa

  Copy full SHA for 86896aa

• server: tests: ci : fix step id duplicated

  phymbert

  committed
  334902b

  Copy full SHA for 334902b

• server: tests: ci : fix cuda install

  phymbert

  committed
  fce2e00

  Copy full SHA for fce2e00

• server: test: ci fix cuda build

  phymbert

  committed
  e4fb790

  Copy full SHA for e4fb790

• server: test: ci fix cublas build

  phymbert

  committed
  2edd995

  Copy full SHA for 2edd995

• server: test: ci fix matrix

  phymbert

  committed
  fa51bac

  Copy full SHA for fa51bac

• server: test: ci fix clblast

  phymbert

  committed
  606738e

  Copy full SHA for 606738e

• server: test: ci fix openblas build

  phymbert

  committed
  d159e29

  Copy full SHA for d159e29

• server: test: ci matrix

  phymbert

  committed
  13863ef

  Copy full SHA for 13863ef

• server: test: ci matrix, experimental on matrix avx512 entry which fail test

  phymbert

  committed
  4d3791a

  Copy full SHA for 4d3791a

• server: test: ci cmake remove all warning as it is done by the classical build and does not matter for testing

  phymbert

  committed
  b94809b

  Copy full SHA for b94809b

• server: test: ci make arch not available pass the test

  phymbert

  committed
  5a621e7

  Copy full SHA for 5a621e7

• server: test: ax512 experimental

  phymbert

  committed
  54ea4d4

  Copy full SHA for 54ea4d4

• server: test: display server logs in case of failure

  phymbert

  committed
  5b2ce45

  Copy full SHA for 5b2ce45

• server: test: fix CUDA LD PATH

  phymbert

  committed
  6dc3af5

  Copy full SHA for 6dc3af5

• server: test: ci debug LD path

  phymbert

  committed
  83c386f

  Copy full SHA for 83c386f

• server: test: ci debug CI LD path

  phymbert

  committed
  0d380ae

  Copy full SHA for 0d380ae

• server: test: ci switch to nvidia based docker image for cuda

  phymbert

  committed
  c75e0e1

  Copy full SHA for c75e0e1

• server: test: ci give up with nvidia as it requires the nvidia docker runtime

  phymbert

  committed
  2c8bf24

  Copy full SHA for 2c8bf24

• server: test: ci rename step name to Test, change matrix order for better clarity

  phymbert

  committed
  777bdcf

  Copy full SHA for 777bdcf

• server: test: ci rename job name to Server

  phymbert

  committed
  e10b83a

  Copy full SHA for e10b83a

• server: tests: move all requests call to asyncio

  phymbert

  committed
  4d27466

  Copy full SHA for 4d27466

• server: tests: allow to pass argument to the test file

  Show description for 1c1fd40
  phymbert

  committed
  1c1fd40

  Copy full SHA for 1c1fd40

• server: tests: print server logs only on github action

  phymbert

  committed
  2109743

  Copy full SHA for 2109743

• server: tests: check if the server has not crashed after a scenario

  phymbert

  committed
  30f802d

  Copy full SHA for 30f802d

• server: tests: adding concurrent embedding in issue ggerganov#5655

  Show description for 6c0e6f4
  phymbert

  committed
  6c0e6f4

  Copy full SHA for 6c0e6f4

• server: tests: linter

  phymbert

  committed
  77b8589

  Copy full SHA for 77b8589

• server: tests: fix concurrent OAI streaming request

  phymbert

  committed
  7183149

  Copy full SHA for 7183149

• server: tests: add a note regarding inference speed.

  phymbert

  committed
  2d107ba

  Copy full SHA for 2d107ba

Commits on Feb 24, 2024

• server: tests: removing debug print

  phymbert

  committed
  124ca77

  Copy full SHA for 124ca77

• server: tests - allow print on debug

  phymbert

  committed
  5957a2d

  Copy full SHA for 5957a2d

• server: tests - README.md add build instruction and notice on @bug and @wrong_usage.

  phymbert

  committed
  482eb30

  Copy full SHA for 482eb30

• server: tests - add explanation about KV Cache.

  phymbert

  committed
  60781f0

  Copy full SHA for 60781f0

• server: tests - print only in case of DEBUG

  phymbert

  committed
  a779a4b

  Copy full SHA for a779a4b

• server: add link to tests in the README.md

  phymbert

  committed
  a2a928c

  Copy full SHA for a2a928c

• server: tests: improved README.md

  phymbert

  committed
  5ed4452

  Copy full SHA for 5ed4452

• github issue template: add link to the tests server framework

  phymbert

  committed
  99163c8

  Copy full SHA for 99163c8