Skip to main content
Version: 1.54

Supported versions

This article describes the supported versions and compatibility with supported ML frameworks. For some frameworks, links to example repositories on Gitlab are included. These example repositories can be used to create example deployments on Deeploy.

Deeploy version1.38.0 - 1.41.x1.42.0 - 1.48.01.49.0 - 1.50.11.51.0+
KServe0.12.00.13.10.15.10.15.1
Python3.8 - 3.113.10 - 3.113.9 - 3.113.9 - 3.12

Model frameworks​

The versions described in the following sections are the recommended versions on which the model images have been built. If you deviate from the version, check for potential compatibility issues.

PyTorch​

Pytorch default imagepytorch/torchserve-kfs:0.6.0pytorch/torchserve-kfs:0.8.0pytorch/torchserve-kfs:0.9.0
torch1.7.12.0.02.1.0
torchserve0.7.00.8.00.9.0
torch-model-archiver0.7.00.8.00.9.0

Tensorflow​

Tensorflow default imagetensorflow/serving:2.6.2tensorflow/serving:2.6.2
tensorflow2.6.22.6.2

XGBoost​

XGBoost default imagekserve/xgbserver:v0.12.0kserve/xgbserver:v0.13.1kserve/xgbserver:v0.15.1+
xgboost2.0.32.0.32.1.1

Catboost​

Catboost default imagedeeployml/catboostserver_shaptree:v0.15.1-deeploy-1.54.0
catboost1.2.7
Environment Variables for catboost

You can customize the Catboost model behavior by setting the following environment variables:

  • PREDICTION_TYPE: Controls the output format of predictions

    • RawFormulaVal: Return raw value (default)
    • Class: Return class label
    • Probability: Return probability for every class
    • Exponent: Return exponent of raw formula value
    • RMSEWithUncertainty: Return standard deviation for RMSEWithUncertainty loss function

  • THREAD_COUNT: Number of threads to use (-1 by default)

  • TASK_TYPE: The calcer type

    • CPU
    • GPU

Scikit-learn​

Scikit-learn default imagekserve/sklearnserver:v0.11.0kserve/sklearnserver:v0.13.1kserve/sklearnserver:v0.15.1+
scikit-learn1.3.01.3.01.5.1
joblib1.3.11.3.11.4.0

LightGBM​

LightGBM default imagekserve/lgbserver:v0.11.0kserve/lgbserver:v0.13.1kserve/lgbserver:v0.15.1
lightgbm3.3.23.3.24.5.0

Huggingface​

Triton server default imagekserve/huggingfaceserver:v0.12.0+
huggingface-hub>=0.14.0, <1.0.0

NVIDIA Triton​

Triton server default imagenvcr.io/nvidia/tritonserver:23.05-py3
triton2.34.0
pytorch2.0.0
onnx1.15.0
tensorrt8.6.1.2
tensorflow2.12.0

Custom model Docker images​

Deeploy version1.38.x - 1.41.x1.42.x - 1.48.xCloud & 1.49.0+
deeploy-cli>= 1.38.0>= 1.42.0>= 1.49.0
kserve0.12.00.13.10.15.1

Standard explainers​

Tree SHAP​

Deploy a tree SHAP explainer without training it yourself. Available only for tree-based classification models when using the XGBoost, Scikit-learn, or LightGBM model frameworks.

Saliency​

Deploy a saliency based explainer without training it yourself. Available only for text generation and text-to-text generation models when using the Hugging Face model framework.

  • Supported standard model frameworks: Huggingface
  • Supported schema: completion
  • Original prediction returned in /completion with explain request: yes

Attention​

Deploy an attention based explainer without training it yourself. Available only for text generation and text-to-text generation models when using the Hugging Face model framework.

  • Supported standard model frameworks: Huggingface
  • Supported schema: completion
  • Original prediction returned in /completion with explain request: yes

Trained explainer frameworks​

The versions described in the following sections are the recommended versions for the specific frameworks to use. If you deviate from the version, check for potential compatibility issues. Moreover, for each explainer (version) you can find whether the prediction is returned alongside when making request at the /explain. This can prevent doing redundant requests to obtain prediction separately.

SHAP​

Deeploy version<1.32.0Cloud & 1.32.0+
shap0.36.00.46.0
dill0.3.30.3.7
original prediction returnedyesyes

Anchors​

Deeploy version<1.32.0Cloud & 1.32.0+
alibi0.6.40.9.4
dill0.3.30.3.7
original prediction returnednono

MACE​

Deeploy version<1.32.0Cloud & 1.32.0+
omnixai1.1.41.3.1
dill0.3.30.3.7
original prediction returnednono

PDP​

  • Expected pre-trained explainer artefact: dill (explainer.dill)
Deeploy version<1.32.0Cloud & 1.32.0+
omnixai1.1.41.3.1
dill0.3.30.3.7
original prediction returnednono

Custom explainer Docker images​

Deeploy version1.38.0 - 1.41.x1.41.1.42.x - 1.48.xCloud & 1.49.0+
deeploy-cli>= 1.38.0>= 1.42.0>= 1.49.0
kserve0.12.00.13.10.15.1
original prediction returnedcustomcustomcustom

Transformer frameworks​

Custom transformer Docker images​

Deeploy versionv1.38.0 - 1.41.xv1.42.x - v1.48.xCloud & v1.49.0+
deeploy-cli>= 1.38.0>= 1.42.0>= 1.49.0
kserve0.12.00.13.10.15.1