Skip to main content
Version: Cloud

Supported versions

This article describes the supported versions and compatibility with supported ML frameworks. For some frameworks, links to example repositories on Gitlab are included. These example repositories can be used to create example deployments on Deeploy.

Deeploy version1.38.0 - 1.41.x1.42.0 - 1.48.01.49.0 - 1.50.11.51.0+
KServe0.12.00.13.10.15.10.15.1
Python3.8 - 3.113.10 - 3.113.9 - 3.113.9 - 3.12
Version compatibility

The framework versions listed for model and explainer frameworks are the tested compatible versions, these are the versions that have been verified to work with the corresponding Deeploy and default server images. Other versions may also work, but have not been explicitly tested. If you use a different version, verify compatibility and watch for potential issues.

Model frameworks​

The versions described in the following sections are the recommended versions on which the model serving images have been built.

PyTorch​

Pytorch default imagepytorch/torchserve-kfs:0.6.0pytorch/torchserve-kfs:0.8.0pytorch/torchserve-kfs:0.9.0
torch1.7.12.0.02.1.0
torchserve0.7.00.8.00.9.0
torch-model-archiver0.7.00.8.00.9.0

Tensorflow​

Tensorflow default imagetensorflow/serving:2.6.2tensorflow/serving:2.6.2
tensorflow2.6.22.6.2

XGBoost​

XGBoost default imagekserve/xgbserver:v0.12.0kserve/xgbserver:v0.13.1kserve/xgbserver:v0.15.1+
xgboost2.0.32.0.32.1.1

Catboost​

Catboost default imagedeeployml/catboostserver_shaptree:v0.15.1-deeploy-1.54.0
catboost1.2.7
Environment Variables for catboost

You can customize the Catboost model behavior by setting the following environment variables:

  • PREDICTION_TYPE: Controls the output format of predictions

    • RawFormulaVal: Return raw value (default)
    • Class: Return class label
    • Probability: Return probability for every class
    • Exponent: Return exponent of raw formula value
    • RMSEWithUncertainty: Return standard deviation for RMSEWithUncertainty loss function

  • THREAD_COUNT: Number of threads to use (-1 by default)

  • TASK_TYPE: The calcer type

    • CPU
    • GPU

Scikit-learn​

Scikit-learn default imagekserve/sklearnserver:v0.11.0kserve/sklearnserver:v0.13.1kserve/sklearnserver:v0.15.1+
scikit-learn1.3.01.3.01.5.1
joblib1.3.11.3.11.4.0

LightGBM​

LightGBM default imagekserve/lgbserver:v0.11.0kserve/lgbserver:v0.13.1kserve/lgbserver:v0.15.1
lightgbm3.3.23.3.24.5.0

Huggingface​

Triton server default imagekserve/huggingfaceserver:v0.12.0+
huggingface-hub>=0.14.0, <1.0.0

NVIDIA Triton​

Triton server default imagenvcr.io/nvidia/tritonserver:23.05-py3
triton2.34.0
pytorch2.0.0
onnx1.15.0
tensorrt8.6.1.2
tensorflow2.12.0

Custom model Docker images​

Deeploy version1.38.x - 1.41.x1.42.x - 1.48.xCloud & 1.49.0+
deeploy-cli>= 1.38.0>= 1.42.0>= 1.49.0
kserve0.12.00.13.10.15.1

Standard explainers​

Tree SHAP​

Deploy a tree SHAP explainer without training it yourself. Available only for tree-based classification models when using the XGBoost, Scikit-learn, or LightGBM model frameworks.

Saliency​

Deploy a saliency based explainer without training it yourself. Available only for text generation and text-to-text generation models when using the Hugging Face model framework.

  • Supported standard model frameworks: Huggingface
  • Supported schema: completion
  • Original prediction returned in /completion with explain request: yes

Attention​

Deploy an attention based explainer without training it yourself. Available only for text generation and text-to-text generation models when using the Hugging Face model framework.

  • Supported standard model frameworks: Huggingface
  • Supported schema: completion
  • Original prediction returned in /completion with explain request: yes

Trained explainer frameworks​

The versions described in the following sections are the recommended versions for the specific frameworks to use. If you deviate from the version, check for potential compatibility issues. Moreover, for each explainer (version) you can find whether the prediction is returned alongside when making request at the /explain. This can prevent doing redundant requests to obtain prediction separately.

SHAP​

Deeploy version<1.32.0Cloud & 1.32.0+
shap0.36.00.46.0
dill0.3.30.3.7
original prediction returnedyesyes

Anchors​

Deeploy version<1.32.0Cloud & 1.32.0+
alibi0.6.40.9.4
dill0.3.30.3.7
original prediction returnednono

MACE​

Deeploy version<1.32.0Cloud & 1.32.0+
omnixai1.1.41.3.1
dill0.3.30.3.7
original prediction returnednono

PDP​

  • Expected pre-trained explainer artefact: dill (explainer.dill)
Deeploy version<1.32.0Cloud & 1.32.0+
omnixai1.1.41.3.1
dill0.3.30.3.7
original prediction returnednono

Custom explainer Docker images​

Deeploy version1.38.0 - 1.41.x1.41.1.42.x - 1.48.xCloud & 1.49.0+
deeploy-cli>= 1.38.0>= 1.42.0>= 1.49.0
kserve0.12.00.13.10.15.1
original prediction returnedcustomcustomcustom

Transformer frameworks​

Custom transformer Docker images​

Deeploy versionv1.38.0 - 1.41.xv1.42.x - v1.48.xCloud & v1.49.0+
deeploy-cli>= 1.38.0>= 1.42.0>= 1.49.0
kserve0.12.00.13.10.15.1