Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Python backend based PyTorch backend documentations #94

Merged
merged 2 commits into from
Jan 11, 2024
Merged

Python backend based PyTorch backend documentations #94

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
05ff5c0
Update docs
kthui Jan 8, 2024
48af0d0
Update copyright
kthui Jan 8, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 12 additions & 3 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
<!--
# Copyright 2021-2023, NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# Copyright 2021-2024, NVIDIA CORPORATION & AFFILIATES. All rights reserved.
#
# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions
Expand Down Expand Up @@ -80,8 +80,8 @@ used to execute both TensorFlow 1 and TensorFlow 2 models. The
[tensorflow_backend](https://github.com/triton-inference-server/tensorflow_backend)
repo contains the documentation and source for the backend.

**PyTorch**: The PyTorch backend is used to execute TorchScript
models. The
**PyTorch**: The PyTorch backend is used to execute PyTorch models in both
TorchScript and PyTorch 2.0 formats. The
[pytorch_backend](https://github.com/triton-inference-server/pytorch_backend)
repo contains the documentation and source for the backend.

Expand Down Expand Up @@ -215,6 +215,15 @@ Triton image as:
... # other files needed by mybackend
```

Starting from 24.01, the default backend shared library name can be changed by
providing the `runtime` setting in the model configuration. For example,

```
runtime: "my_backend_shared_library_name.so"
```

A model may choose a specific runtime implementation provided by the backend.
rmccorm4 marked this conversation as resolved.
Show resolved Hide resolved

### Triton Backend API

A Triton backend must implement the C interface defined in
Expand Down