Skip to content

Commit

Permalink
Merge branch 'master' into scrums/1
Browse files Browse the repository at this point in the history
  • Loading branch information
@bdattoma
bdattoma authored Jun 28, 2024
2 parents 6c837be + ce39b20 commit c81fb6c
Show file tree
Hide file tree
Showing 9 changed files with 270 additions and 25 deletions.
5 changes: 3 additions & 2 deletions ods_ci/tests/Resources/CLI/ModelServing/llm.resource
View file
Original file line number Diff line number Diff line change
Expand Up @@ -321,6 +321,7 @@ Query Model Multiple Times
... ${runtime}=caikit-tgis-runtime
... ${isvc_name}=${model_name}
... ${inference_type}=all-tokens
... ${status_code}=200
... ${n_times}=10
... ${query_idx}=0
... ${validate_response}=${TRUE}
Expand Down Expand Up @@ -356,7 +357,7 @@ Query Model Multiple Times
... inference_type=${inference_type} model_name=${model_name} body_params=${body_params}
... query_text=${EXP_RESPONSES}[queries][${query_idx}][query_text]
IF "${token}" != "${None}"
${header}= Set Variable "Authorization: Bearer ${token}" -H ${header}
Set To Dictionary ${header} Authorization Bearer ${token}
END
${runtime_details}= Set Variable ${RUNTIME_FORMATS}[${runtime}][endpoints][${inference_type}][${protocol}]
${endpoint}= Set Variable ${runtime_details}[endpoint]
Expand All @@ -381,7 +382,7 @@ Query Model Multiple Times
Log Dictionary ${args}
${res}= Run Keyword And Continue On Failure Perform Request request_type=POST
... skip_res_json=${skip_json_load_response} &{args}
Run Keyword And Continue On Failure Status Should Be 200
Run Keyword And Continue On Failure Status Should Be ${status_code}
END
Log ${res}
IF ${validate_response} == ${TRUE}
Expand Down
53 changes: 45 additions & 8 deletions ods_ci/tests/Resources/Page/ODH/ODHDashboard/ODHDataScienceProject/ModelServer.resource
View file
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,8 @@ ${SERVING_ACCELERATOR_MINUS_BUTTON_XPATH}= xpath=${SERVING_ACCELERATOR_INPUT_
${SERVING_ACCELERATOR_PLUS_BUTTON_XPATH}= xpath=${SERVING_ACCELERATOR_INPUT_XPATH}/following-sibling::button
${SERVING_MODEL_SERVERS_SIDE_MENU}= xpath=//span[text()='Models and model servers']
${TOKEN_AUTH_CHECKBOX_XP}= xpath://input[@id="alt-form-checkbox-auth"]
${ADD_SERVICE_ACCOUNT_BUTTON}= xpath://button[text()='Add a service account']
${SERVICE_ACCOUNT_INPUT}= xpath://input[@data-testid='service-account-form-name']
${REPLICAS_COUNT_XP}= xpath://section[@class='pf-v5-c-form__section']//span[@class='pf-v5-c-form-control']/input


Expand Down Expand Up @@ -176,22 +178,56 @@ Enable External Serving Route

Enable Token Authentication
[Documentation] Enables Token authentication to serving route
[Arguments] ${service_account_name}=${NONE}
SeleniumLibrary.Select Checkbox ${TOKEN_AUTH_CHECKBOX_XP}
# TODO: change service account name
IF "${service_account_name}" != "${NONE}"
Input Service Account Name ${service_account_name}
END

Disable Token Authentication
[Documentation] Enables Token authentication to serving route
SeleniumLibrary.Unselect Checkbox ${TOKEN_AUTH_CHECKBOX_XP}
# TODO: change service account name
[Documentation] Disable Token authentication to serving route
... If a Service Account is given, just disable the token for the specific one.
[Arguments] ${service_account_name}=${NONE}
IF "${service_account_name}" != "${NONE}"
Click Button xpath://input[@value="${service_account_name}"]/parent::span/parent::div/following::div/button[@aria-label="Remove service account"]
ELSE
SeleniumLibrary.Unselect Checkbox ${TOKEN_AUTH_CHECKBOX_XP}
END

Add Another Service Account
[Documentation] Add another Service Account once the Token Authorization is enabled
[Arguments] ${service_account_name}=default-name2
Click Button ${ADD_SERVICE_ACCOUNT_BUTTON}
Input Service Account Name ${service_account_name}

Input Service Account Name
[Documentation] Input the name of the last Added Service Account.
[Arguments] ${service_account_name}=default-name
${service_accounts}= Get WebElements ${SERVICE_ACCOUNT_INPUT}
${last_service_account}= Set Variable ${service_accounts}[-1]
Clear Element And Input Text ${last_service_account} ${service_account_name}

Get Model Serving Access Token via UI
[Documentation] Returns the token used for authentication to the serving route
... TODO: There can be multiple tokens defined for each model server, handle this case as well
[Arguments] ${service_account_name}=default-name
Wait Until Page Contains Element xpath://td[@data-label="Tokens"]/button
Click Element xpath://td[@data-label="Tokens"]/button
[Arguments] ${service_account_name}=default-name ${single_model}=${FALSE} ${model_name}=${NONE}
IF ${single_model}
# Expand the model
SeleniumLibrary.Click Button xpath: //div[text()='${model_name}']/parent::td[@data-label='Name']/preceding-sibling::td/button
${token}= Get Single Model Token ${service_account_name}
ELSE
Wait Until Page Contains Element xpath://td[@data-label="Tokens"]/button
Click Element xpath://td[@data-label="Tokens"]/button
${token}= Get Element Attribute
... xpath://div[.="${service_account_name} "]/../../td[@data-label="Token Secret"]//input value
END
RETURN ${token}

Get Single Model Token
[Documentation] Gets the (Single Model) Token of an specific Service Account
[Arguments] ${service_account_name}
${token}= Get Element Attribute
... xpath://div[.="${service_account_name} "]/../../td[@data-label="Token Secret"]//input value
... xpath://div[text()='${service_account_name}']/parent::td[@data-label='Token Name']/following-sibling::td[@data-label='Token Secret']//input value
RETURN ${token}

Click Plus Button
Expand Down Expand Up @@ -274,6 +310,7 @@ Switch Model Serving Project
IF "${project_name}" == "${NONE}"
${project_name}= Set Variable All projects
END
Wait Until Element Is Visible //main[contains(@id, 'dashboard-page-main')]//button[contains(@id, 'dropdown-toggle')]
Click Element //main[contains(@id, 'dashboard-page-main')]//button[contains(@id, 'dropdown-toggle')]
Wait Until Element Is Visible //ul//a[contains(., '${project_name}')]
Click Element //ul//a[contains(., '${project_name}')]
Expand Down
19 changes: 14 additions & 5 deletions ods_ci/tests/Resources/Page/ODH/ODHDashboard/ODHModelServing.resource
View file
Original file line number Diff line number Diff line change
Expand Up @@ -200,7 +200,8 @@ Open Model Edit Modal
[Documentation] Opens the modal used to edit the details of an already deployed model.
[Arguments] ${model_name}
ODHDashboard.Click Action From Actions Menu item_title=${model_name} action=Edit
Wait For Dashboard Page Title Deploy model
# Wait for modal header
Wait Until Page Contains Element xpath://span[contains(@class, "modal") and text()="Deploy model"]

Get Model Route Via UI
[Documentation] Grabs the serving route (URL) of an already deployed model from the Model Serving page.
Expand Down Expand Up @@ -238,12 +239,13 @@ Open ${section} Options Menu
SeleniumLibrary.Click Element xpath://span[.="${section}"]/../../..//button # Remove attribute which is not present anymore
END

Get Access Token via UI
Get Access Token Via UI
[Documentation] Returns the access token for models deployed in a specific project
... by using the UI of DSP
[Arguments] ${project_name} ${service_account_name}=default-name
[Arguments] ${project_name} ${service_account_name}=default-name ${single_model}=${FALSE}
... ${model_name}=${NONE}
Open Data Science Project Details Page ${project_name} tab_id=model-server
${token}= Get Model Serving Access Token via UI ${service_account_name}
${token}= Get Model Serving Access Token via UI ${service_account_name} ${single_model} ${model_name}
RETURN ${token}

Get Model Project
Expand All @@ -269,7 +271,7 @@ Get Model Inference
IF "${project_title}" == "${NONE}"
${project_title}= Get Model Project ${model_name}
END
${token}= Get Access Token via UI ${project_title}
${token}= Get Access Token Via UI ${project_title}
${curl_cmd}= Catenate ${curl_cmd} -H "Authorization: Bearer ${token}"
END
IF ${kserve}
Expand Down Expand Up @@ -388,6 +390,7 @@ Deploy Kserve Model Via UI #robocop: disable
[Documentation] Deploys a model using the kserve/caikit runtime using the Data Science Projects UI
[Arguments] ${model_name} ${serving_runtime} ${data_connection} ${path}
... ${model_framework}=caikit ${replicas}=1 ${size}=Small ${no_gpus}=${0}
... ${token}=${FALSE} ${multi_token}=${FALSE} ${multi_service_account_name}=default-name2
Move To Tab Models
Click Button ${DEPLOY_MODEL_BTN}
Wait Until Page Contains Element xpath=${KSERVE_MODAL_HEADER}
Expand All @@ -396,6 +399,12 @@ Deploy Kserve Model Via UI #robocop: disable
Select Framework ${model_framework}
Set Replicas Number With Buttons ${replicas}
Set Server Size ${size}
IF ${token}
Enable Token Authentication
IF ${multi_token}
Add Another Service Account ${multi_service_account_name}
END
END
Select Existing Data Connection ${data_connection}
Set Folder Path ${path}
IF ${no_gpus} > ${0}
Expand Down
128 changes: 128 additions & 0 deletions .../tests/Tests/1000__model_serving/1007__model_serving_llm/1007__model_serving_llm_UI.robot
View file
Original file line number Diff line number Diff line change
Expand Up @@ -195,6 +195,134 @@ Verify Non Admin Can Serve And Query A Model Using The UI # robocop: disable
... namespace=${test_namespace} protocol=http validate_response=$FALSE
[Teardown] Run Keywords Clean Up DSP Page AND Close Browser AND Switch Browser 1

Verify User Can Serve And Query A Token Protected Model Using The UI
[Documentation] Deploying and querying a Token Protected LLM model
... using Kserve and Caikit runtime
[Tags] Tier1 RHOAIENG-4603
[Setup] Set Up Project namespace=${TEST_NS}
${test_namespace}= Set Variable ${TEST_NS}
${flan_model_name}= Set Variable flan-t5-small-caikit
Deploy Kserve Model Via UI ${flan_model_name} serving_runtime=Caikit TGIS data_connection=kserve-connection
... path=flan-t5-small/${flan_model_name} token=${TRUE}
Wait For Model KServe Deployment To Be Ready label_selector=serving.kserve.io/inferenceservice=${flan_model_name}
... namespace=${test_namespace} runtime=${CAIKIT_TGIS_RUNTIME_NAME}
${model_token}= Get Access Token Via UI ${test_namespace} single_model=${TRUE}
... model_name=${flan_model_name}
Query Model Multiple Times model_name=${flan_model_name}
... inference_type=all-tokens n_times=1
... namespace=${test_namespace} protocol=http
... token=${model_token}
Delete Model Via UI ${flan_model_name}
[Teardown] Clean Up DSP Page

Verify User Can Serve But Can't Query A Token Protected Model Without The Token
[Documentation] Deploying and querying a Token Protected LLM model
... using Kserve and Caikit runtime
[Tags] Tier1 RHOAIENG-4603
[Setup] Set Up Project namespace=${TEST_NS}
${test_namespace}= Set Variable ${TEST_NS}
${flan_model_name}= Set Variable flan-t5-small-caikit
Deploy Kserve Model Via UI ${flan_model_name} serving_runtime=Caikit TGIS data_connection=kserve-connection
... path=flan-t5-small/${flan_model_name} token=${TRUE}
Wait For Model KServe Deployment To Be Ready label_selector=serving.kserve.io/inferenceservice=${flan_model_name}
... namespace=${test_namespace} runtime=${CAIKIT_TGIS_RUNTIME_NAME}
Query Model Multiple Times model_name=${flan_model_name}
... inference_type=all-tokens status_code=401 n_times=1 validate_response=${FALSE}
... namespace=${test_namespace} protocol=http
Delete Model Via UI ${flan_model_name}
[Teardown] Clean Up DSP Page

Verify User Can Serve And Query A Model Using The UI Protected With Multiple Tokens
[Documentation] Deploying and querying a Token Protected LLM model
... using Kserve and Caikit runtime, using multiple tokens
[Tags] Tier1 RHOAIENG-4603
[Setup] Set Up Project namespace=${TEST_NS}
${test_namespace}= Set Variable ${TEST_NS}
${flan_model_name}= Set Variable flan-t5-small-caikit
Deploy Kserve Model Via UI ${flan_model_name} serving_runtime=Caikit TGIS data_connection=kserve-connection
... path=flan-t5-small/${flan_model_name} token=${TRUE} multi_token=${TRUE}
Wait For Model KServe Deployment To Be Ready label_selector=serving.kserve.io/inferenceservice=${flan_model_name}
... namespace=${test_namespace} runtime=${CAIKIT_TGIS_RUNTIME_NAME}
${model_token_1}= Get Access Token Via UI ${test_namespace} service_account_name=default-name
... single_model=${TRUE} model_name=${flan_model_name}
${model_token_2}= Get Access Token Via UI ${test_namespace} service_account_name=default-name2
... single_model=${TRUE} model_name=${flan_model_name}
Query Model Multiple Times model_name=${flan_model_name}
... inference_type=all-tokens n_times=1
... namespace=${test_namespace} protocol=http
... token=${model_token_1}
Query Model Multiple Times model_name=${flan_model_name}
... inference_type=all-tokens n_times=1
... namespace=${test_namespace} protocol=http
... token=${model_token_2}
Delete Model Via UI ${flan_model_name}
[Teardown] Clean Up DSP Page

Verify User Can not Query A Token Protected Model With A Disabled Token Using The UI
[Documentation] Deploy an LLM (using Kserve and Caikit runtime) protected by different Tokens
... and try to query it using a disabled token
[Tags] Tier1 RHOAIENG-4603
[Setup] Set Up Project namespace=${TEST_NS}
${test_namespace}= Set Variable ${TEST_NS}
${flan_model_name}= Set Variable flan-t5-small-caikit
Deploy Kserve Model Via UI ${flan_model_name} serving_runtime=Caikit TGIS data_connection=kserve-connection
... path=flan-t5-small/${flan_model_name} token=${TRUE} multi_token=${TRUE}
Wait For Model KServe Deployment To Be Ready label_selector=serving.kserve.io/inferenceservice=${flan_model_name}
... namespace=${test_namespace} runtime=${CAIKIT_TGIS_RUNTIME_NAME}
${model_token_1}= Get Access Token Via UI ${test_namespace} service_account_name=default-name
... single_model=${TRUE} model_name=${flan_model_name}
Open Model Edit Modal ${flan_model_name}
Disable Token Authentication service_account_name=default-name
Click Button Deploy
Wait For Model KServe Deployment To Be Ready label_selector=serving.kserve.io/inferenceservice=${flan_model_name}
... namespace=${test_namespace} runtime=${CAIKIT_TGIS_RUNTIME_NAME}
Run Keyword And Expect Error *Expected status: 401 != 200*
... Query Model Multiple Times model_name=${flan_model_name}
... inference_type=all-tokens n_times=1
... namespace=${test_namespace} protocol=http
... token=${model_token_1}
Delete Model Via UI ${flan_model_name}
[Teardown] Clean Up DSP Page

Verify User Can Query A Token Protected Model Using The UI Without Token After Disabling Token Protection
[Documentation] Deploying and querying a Token Protected LLM model using Kserve and Caikit runtime.
... Verify the token Works and without Token it Does not Work.
... Disable the token authentication and verify that the model can be queried without token.
[Tags] Sanity Tier1 RHOAIENG-4603
[Setup] Set Up Project namespace=${TEST_NS}
${test_namespace}= Set Variable ${TEST_NS}
${flan_model_name}= Set Variable flan-t5-small-caikit
Deploy Kserve Model Via UI ${flan_model_name} serving_runtime=Caikit TGIS data_connection=kserve-connection
... path=flan-t5-small/${flan_model_name} token=${TRUE}
Wait For Model KServe Deployment To Be Ready label_selector=serving.kserve.io/inferenceservice=${flan_model_name}
... namespace=${test_namespace} runtime=${CAIKIT_TGIS_RUNTIME_NAME}
${model_token}= Get Access Token Via UI ${test_namespace} single_model=${TRUE}
... model_name=${flan_model_name}
Query Model Multiple Times model_name=${flan_model_name}
... inference_type=all-tokens n_times=1
... namespace=${test_namespace} protocol=http
... token=${model_token}
Run Keyword And Expect Error *Expected status: 401 != 200*
... Query Model Multiple Times model_name=${flan_model_name}
... inference_type=all-tokens n_times=1
... namespace=${test_namespace} protocol=http
Open Model Edit Modal ${flan_model_name}
Disable Token Authentication
Click Button Deploy
# The verification will fail due to a temporary duplication of the replicas while the model is restarted
Sleep 60s msg=Wait for the model pod replicas to scale down
Wait For Pods Numbers 1
... namespace=${test_namespace}
... label_selector=serving.kserve.io/inferenceservice=${flan_model_name}
... timeout=1200
Wait For Model KServe Deployment To Be Ready label_selector=serving.kserve.io/inferenceservice=${flan_model_name}
... namespace=${test_namespace} runtime=${CAIKIT_TGIS_RUNTIME_NAME}
Query Model Multiple Times model_name=${flan_model_name}
... inference_type=all-tokens n_times=1
... namespace=${test_namespace} protocol=http
Delete Model Via UI ${flan_model_name}
[Teardown] Clean Up DSP Page

Verify User Can Serve And Query Flan-t5 Grammar Syntax Corrector Using The UI # robocop: disable
[Documentation] Deploys and queries flan-t5-large-grammar-synthesis model
[Tags] Tier2 ODS-2553
Expand Down
11 changes: 9 additions & 2 deletions ...ts/Tests/1000__model_serving/1007__model_serving_llm/1007__model_serving_llm_models.robot
View file
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ ${KSERVE_MODE}= RawDeployment #Serverless
${MODEL_FORMAT}= pytorch #vLLM
${PROTOCOL}= grpc #http
${OVERLAY}= ${EMPTY} #vllm

${GPU_TYPE}= NVIDIA

*** Test Cases ***
Verify User Can Serve And Query A bigscience/mt0-xxl Model
Expand Down Expand Up @@ -845,7 +845,14 @@ Setup Test Variables
Set Test Variable ${storage_uri} s3://${S3.BUCKET_3.NAME}/${model_path}
END
IF ${use_gpu}
${limits}= Create Dictionary nvidia.com/gpu=1
${supported_gpu_type}= Convert To Lowercase ${GPU_TYPE}
IF "${supported_gpu_type}" == "nvidia"
${limits}= Create Dictionary nvidia.com/gpu=1
ELSE IF "${supported_gpu_type}" == "amd"
${limits}= Create Dictionary amd.com/gpu=1
ELSE
FAIL msg=Provided GPU type is not yet supported. Only nvidia and amd gpu type are supported
END
Set Test Variable ${limits}
ELSE
Set Test Variable ${limits} &{EMPTY}
Expand Down
3 changes: 2 additions & 1 deletion ods_ci/tests/Tests/100__deploy/100__installation/112__model_registry.robot
View file
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,8 @@ Verify Model Registry Operator Installation
... deployed in the ${APPLICATIONS_NAMESPACE} namespace in ODS
[Tags] Smoke
... Tier1
... Operator
... ModelRegistry
... OpenDataHub robot:recursive-continue-on-failure
Wait Until Keyword Succeeds 1 min 10 sec Verify Model Registry Operator Deployment
Wait Until Keyword Succeeds 10 times 5s Verify Model Registry ReplicaSets Info
Expand Down Expand Up @@ -52,7 +54,6 @@ Registry Suite Setup
IF "${REGISTRY}" == "false" Enable Component modelregistry
Set Suite Variable ${REGISTRY}


Teardown
[Documentation] Disable Registry if Enabled
SeleniumLibrary.Close Browser
Expand Down
Loading

0 comments on commit c81fb6c

Please sign in to comment.