Skip to content

DJL v0.13.0 release

Compare
Choose a tag to compare
@frankfliu frankfliu released this 12 Oct 19:17
· 1338 commits to master since this release

DJL v0.13.0 brings the new TensorRT and Python engines, and updates the engines PyTorch to 1.9.0, ONNXRuntime to 1.9.0, PaddlePaddle to 2.0.2, and introduces several new features:

Key Features

Enhancement

  • Introduces several new features in djl-serving:
    • Improves djl-serving API to make it easy to get HTTP headers (#1134)
    • Loads models on all GPUs at startup for djl-serving (#1132)
    • Enables asynchronous logging for djl-serving
    • Makes djl-serving access log in separate log file (#1150)
    • Adds configuration to support number worker threads for GPU inference (#1153)
    • Improves auto-scale algorithm for djl-serving (#1149)
  • Introduces several new features in djl-bench:
    • Adds a command line option to djl-bench to generate NDList file (#1155)
    • Adds warmup to benchmark (#1152)
    • Improves djll-bench to support djl:// urls (#1146)
    • Adds support to benchmark on multiple GPUs (#1144)
    • Adds support to benchmark onnx on GPU machines (#1148)
    • Adds support to benchmark TensorRT models (#1257)
    • Adds support to benchmark Python models (#1267)
  • Introduces several new features in PyTorch engine:
    • Supports PyTorch custom input data type with IValue (#1208)
  • Introduces several new features in OnnxRuntime:
    • Adds UINT8 support for OnnxRuntime (#1271)
  • Introduces several new features in PaddlePaddle:
    • Adds more model loading options for PaddlePaddle (#1173)
    • Adds load functionalities to PaddlePaddle (#1140)
    • Adds remove pass option to PaddlePaddle (#1141)
  • Introduces several API improvements:
    • Adds missing NDList.get(String) API (#1194)
    • Adds support to directly load models from a TFHub url (#1231)
    • Improves repository API to support passing argument in the URL query string (#1139)
    • Avoids loading the default engine if it is not being used (#1136)
    • Improves IO by adding a buffer to read/write (#1135)
    • Improves NDArray.toString() debug mode performance (#1142)
    • Makes GPU device detection engine specific to avoid confusion when using multiple engines (#1138)

Documentation and examples

  • Adds Style Transfer example with CycleGAN (#1180)

Breaking change

  • Removes support for Apache MXNet 1.6.0
  • Deprecates Device.getDevices() API - Use Engine.getDevices() instead
  • Renames SimpleVocabulary to DefaultVocabulary

Bug Fixes

  • Fixes broken link in documents
  • Fixes TensorFlow NDArray was created on CPU instead of GPU bug (#1279)
  • Fixes default image processing pipeline (#1268)
  • Fixed XGBoost NDArray multiple read bug (#1239)
  • Fixes platform matching bug (#1167)
  • Fixes NullPointerException in NDArray.toString() (#1157)
  • Fixes PaddlePaddle crash due to GC (#1162)
  • Fixes NDArrayAdapter.getSparseFormat() unsupported bug (#1151)
  • Fixes mixed device issue in multiple engine use case (#1123)
  • Fixes handle duplicate plugin issue for djl-serving (#1108)
  • Fixes XGBoost NDArray creation bug (#1109)
  • Fixes runtime exception running benchmark arm machine(#1107)
  • Fixes unregister model regression (#1101)

Contributors

This release is thanks to the following contributors: