Enable triton-client python interface #8943

smuzaffar · 2024-01-16T13:05:39Z

This enables the triton client python interface. Code like [a] runs in cmssw env with this change.

[a]

bash-4.4$ python3
Python 3.9.14 (main, Jun 21 2023, 12:13:52) 
[GCC 12.3.1 20230527] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from tritonclient import grpc
>>> from google.protobuf import text_format, message, descriptor
>>> model = grpc.model_config_pb2.ModelConfig()
>>> config='/cvmfs/cms.cern.ch/share/cms/data-HeterogeneousCore-SonicTriton/V00-02-00/HeterogeneousCore/SonicTriton/data/models/gat_test/config.pbtxt'
>>> with open(config,'r') as infile:
...   text_format.Parse(infile.read(), model)
... 
name: "gat_test"
platform: "pytorch_libtorch"
input {
  name: "x__0"
  data_type: TYPE_FP32
  dims: -1
  dims: 1433
}
input {
  name: "edgeindex__1"
  data_type: TYPE_INT64
  dims: 2
  dims: -1
}
output {
  name: "logits__0"
  data_type: TYPE_FP32
  dims: -1
  dims: 7
}

>>>

cmsbuild · 2024-01-16T13:06:04Z

A new Pull Request was created by @smuzaffar (Malik Shahzad Muzaffar) for branch IB/CMSSW_14_0_X/master.

@smuzaffar, @iarspider, @aandvalenzuela, @cmsbuild can you please review it and eventually sign? Thanks.
@antoniovilela, @sextonkennedy, @rappoccio you are the release manager for this.
cms-bot commands are listed here

cmsbuild · 2024-01-16T13:06:05Z

cms-bot internal usage

smuzaffar · 2024-01-16T13:12:41Z

please test

smuzaffar · 2024-01-16T14:14:34Z

please test for el8_aarch64_gcc12

smuzaffar · 2024-01-16T14:14:58Z

please test for el8_ppc64le_gcc12

cmsbuild · 2024-01-16T15:37:50Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-edb4ab/36845/summary.html
COMMIT: ea84576
CMSSW: CMSSW_14_0_X_2024-01-15-2300/el8_amd64_gcc12
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmsdist/8943/36845/install.sh to create a dev area with all the needed externals and cmssw changes.

The following merge commits were also included on top of IB + this PR after doing git cms-merge-topic:

You can see more details here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-edb4ab/36845/git-recent-commits.json
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-edb4ab/36845/git-merge-result

Comparison Summary

Summary:

You potentially added 14078 lines to the logs
Reco comparison results: 0 differences found in the comparisons
DQMHistoTests: Total files compared: 10
DQMHistoTests: Total histograms compared: 327017
DQMHistoTests: Total failures: 0
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 327009
DQMHistoTests: Total skipped: 8
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 9 files compared)
Checked 149 log files, 71 edm output root files, 10 DQM output files
TriggerResults: no differences found

kpedro88 · 2024-01-16T16:01:45Z

triton-inference-client.spec

@@ -44,25 +47,46 @@ cmake ${PROJ_DIR} \
    -DCMAKE_CXX_STANDARD=%{cms_cxx_standard} \
    -DTRITON_ENABLE_CC_HTTP=OFF \
    -DTRITON_ENABLE_CC_GRPC=ON \
-    -DTRITON_ENABLE_PYTHON_HTTP=OFF \


If this part is only for building the C++ client (since there's a separate buildpy below), shouldn't these lines be kept here? (the defaults for all of these are OFF, right now, but I think it's better to be explicit, in case the default changes...)

These TRITON_ENABLE_PYTHON_* flags are not available for triton/src/c++/CMakeLists.txt ( which we use to build the c++ interface) and cmake generate warning message that these are unused flags.

Ah, that is because they are in the top-level CMakeLists.txt, but we're using the project-specific ones for more control. Okay, never mind then.

kpedro88 · 2024-01-16T16:02:21Z

triton-inference-client.spec

+    -DCMAKE_INSTALL_LIBDIR=lib \
+    -DCMAKE_BUILD_TYPE=Release \
+    -DCMAKE_CXX_STANDARD=%{cms_cxx_standard} \
+    -DTRITON_ENABLE_PYTHON_HTTP=OFF \


Following the above comment, I would include the corresponding lines here:

-DTRITON_ENABLE_CC_HTTP=OFF \ -DTRITON_ENABLE_CC_GRPC=OFF \

these flags are not available for triton/src/python/CMakeLists.txt which we use to build the python interface

kpedro88 · 2024-01-16T16:02:55Z

thanks @smuzaffar ! Indeed, I think it was the lack of a rapidJSON external that had kept me from enabling the python client.

cmsbuild · 2024-01-16T16:34:05Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-edb4ab/36849/summary.html
COMMIT: ea84576
CMSSW: CMSSW_14_0_X_2024-01-15-2300/el8_ppc64le_gcc12
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmsdist/8943/36849/install.sh to create a dev area with all the needed externals and cmssw changes.

The following merge commits were also included on top of IB + this PR after doing git cms-merge-topic:

You can see more details here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-edb4ab/36849/git-recent-commits.json
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-edb4ab/36849/git-merge-result

cmsbuild · 2024-01-16T16:56:07Z

-1

Failed Tests: RelVals
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-edb4ab/36850/summary.html
COMMIT: ea84576
CMSSW: CMSSW_14_0_X_2024-01-15-2300/el8_aarch64_gcc12
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmsdist/8943/36850/install.sh to create a dev area with all the needed externals and cmssw changes.

The following merge commits were also included on top of IB + this PR after doing git cms-merge-topic:

You can see more details here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-edb4ab/36850/git-recent-commits.json
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-edb4ab/36850/git-merge-result

RelVals

12446.0

----- Begin Fatal Exception 16-Jan-2024 16:47:42 CET-----------------------
An exception of category 'FatalRootError' occurred while
   [0] Processing global end Run run: 1
   [1] Calling method for module DQMGenericClient/'postProcessorTrack'
   Additional Info:
      [a] Fatal Root Error: @SUB=Minuit2
VariableMetricBuilder Initial matrix not pos.def.

----- End Fatal Exception -------------------------------------------------

smuzaffar · 2024-01-16T17:38:06Z

@kpedro88 , should we try to update triton client ?

triton client to https://github.com/triton-inference-server/client/tree/r23.03
triton common https://github.com/triton-inference-server/common/tree/r22.12

kpedro88 · 2024-01-16T17:47:01Z

@smuzaffar we do not have any urgent need to update the triton client at the moment, so let's not complicate this PR.

smuzaffar · 2024-01-16T21:35:55Z

+externals

cmsbuild · 2024-01-16T21:36:18Z

This pull request is fully signed and it will be integrated in one of the next IB/CMSSW_14_0_X/master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @rappoccio, @sextonkennedy, @antoniovilela (and backports should be raised in the release meeting by the corresponding L2)

Enable triton-client python interface

ea84576

cmsbuild added tests-pending externals-pending pending-signatures orp-pending labels Jan 16, 2024

smuzaffar mentioned this pull request Jan 16, 2024

Triton model versioning #8935

Closed

cmsbuild added tests-started and removed tests-pending labels Jan 16, 2024

cmsbuild added tests-approved and removed tests-started labels Jan 16, 2024

kpedro88 reviewed Jan 16, 2024

View reviewed changes

smuzaffar merged commit fc0567d into IB/CMSSW_14_0_X/master Jan 16, 2024
20 of 21 checks passed

cmsbuild added externals-approved fully-signed and removed externals-pending pending-signatures labels Jan 16, 2024

kpedro88 mentioned this pull request Jan 16, 2024

Triton model versioning cms-sw/cmssw#43695

Closed

17 tasks

smuzaffar deleted the tritonclient-python branch January 25, 2024 08:42

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Enable triton-client python interface #8943

Enable triton-client python interface #8943

smuzaffar commented Jan 16, 2024 •

edited

Loading

cmsbuild commented Jan 16, 2024

cmsbuild commented Jan 16, 2024 •

edited

Loading

smuzaffar commented Jan 16, 2024

smuzaffar commented Jan 16, 2024

smuzaffar commented Jan 16, 2024

cmsbuild commented Jan 16, 2024

kpedro88 Jan 16, 2024

smuzaffar Jan 16, 2024

kpedro88 Jan 16, 2024

kpedro88 Jan 16, 2024

smuzaffar Jan 16, 2024

kpedro88 commented Jan 16, 2024

cmsbuild commented Jan 16, 2024

cmsbuild commented Jan 16, 2024

smuzaffar commented Jan 16, 2024

kpedro88 commented Jan 16, 2024

smuzaffar commented Jan 16, 2024

cmsbuild commented Jan 16, 2024

Enable triton-client python interface #8943

Enable triton-client python interface #8943

Conversation

smuzaffar commented Jan 16, 2024 • edited Loading

cmsbuild commented Jan 16, 2024

cmsbuild commented Jan 16, 2024 • edited Loading

smuzaffar commented Jan 16, 2024

smuzaffar commented Jan 16, 2024

smuzaffar commented Jan 16, 2024

cmsbuild commented Jan 16, 2024

Comparison Summary

kpedro88 Jan 16, 2024

Choose a reason for hiding this comment

smuzaffar Jan 16, 2024

Choose a reason for hiding this comment

kpedro88 Jan 16, 2024

Choose a reason for hiding this comment

kpedro88 Jan 16, 2024

Choose a reason for hiding this comment

smuzaffar Jan 16, 2024

Choose a reason for hiding this comment

kpedro88 commented Jan 16, 2024

cmsbuild commented Jan 16, 2024

cmsbuild commented Jan 16, 2024

RelVals

smuzaffar commented Jan 16, 2024

kpedro88 commented Jan 16, 2024

smuzaffar commented Jan 16, 2024

cmsbuild commented Jan 16, 2024

smuzaffar commented Jan 16, 2024 •

edited

Loading

cmsbuild commented Jan 16, 2024 •

edited

Loading