Supported versions

This article describes the supported versions and compatibility with supported ML frameworks. For some frameworks, links to example repositories on Gitlab are included. These example repositories can be used to create example deployments on Deeploy.

Deeploy version	1.38.0 - 1.41.x	1.42.0 - 1.48.0	1.49.0+
KServe	0.12.0	0.13.1	0.15.1
Python	3.8 - 3.11	3.10 - 3.11	3.9 - 3.11

Model frameworks

The versions described in the following sections are the recommended versions on which the model images have been built. If you deviate from the version, check for potential compatibility issues.

PyTorch

• Example: https://gitlab.com/deeploy-ml/sample-models/example-pytorch-age-and-gender-classification-with-integrated-explainer
• Expected pre-trained model artefact: Model Archive
• GPUs are supported by default CUDA >=12.1

Pytorch default image	pytorch/torchserve-kfs:0.6.0	pytorch/torchserve-kfs:0.8.0	pytorch/torchserve-kfs:0.9.0
torch	1.7.1	2.0.0	2.1.0
torchserve	0.7.0	0.8.0	0.9.0
torch-model-archiver	0.7.0	0.8.0	0.9.0

Tensorflow

• Example: https://gitlab.com/deeploy-ml/sample-models/imdb
• Expected pre-trained model artefact: SavedModel
• GPUs are supported by default CUDA >=11.2

Tensorflow default image	tensorflow/serving:2.6.2	tensorflow/serving:2.6.2
tensorflow	2.6.2	2.6.2

XGBoost

• Example: https://gitlab.com/deeploy-ml/sample-models/iris-proba
• Expected pre-trained model artefact: Booster (model.json, model.ubj, model.bst)

XGBoost default image	kserve/xgbserver:v0.12.0	kserve/xgbserver:v0.13.1	kserve/xgbserver:v0.15.1+
xgboost	2.0.3	2.0.3	2.1.1

Scikit-learn

• Example: https://gitlab.com/deeploy-ml/sample-models/example-sklearn-census
• Expected pre-trained model artefact: Joblib (model.joblib)

Scikit-learn default image	kserve/sklearnserver:v0.11.0	kserve/sklearnserver:v0.13.1	kserve/sklearnserver:v0.15.1+
scikit-learn	1.3.0	1.3.0	1.5.1
joblib	1.3.1	1.3.1	1.4.0

LightGBM

• Example: https://gitlab.com/deeploy-ml/sample-models/example-iris-lightgbm
• Expected pre-trained model artefact: Booster (model.bst)

LightGBM default image	kserve/lgbserver:v0.11.0	kserve/lgbserver:v0.13.1	kserve/lgbserver:v0.15.1
lightgbm	3.3.2	3.3.2	4.5.0

Huggingface

• Expected model configuration: Huggingface Saved Model or [Huggingface Hub Model ID]
• GPUs supported by default, built on CUDA 12.4.1

Triton server default image	kserve/huggingfaceserver:v0.12.0+
huggingface-hub	>=0.14.0, <1.0.0

NVIDIA Triton

• Expected model configuration: Triton model configuration
• The framework support matrix
• GPUs supported by default, built on CUDA 12.1.1

Triton server default image	nvcr.io/nvidia/tritonserver:23.05-py3
triton	2.34.0
pytorch	2.0.0
onnx	1.15.0
tensorrt	8.6.1.2
tensorflow	2.12.0

Custom model Docker images

• Example: https://gitlab.com/deeploy-ml/sample-models/example-custom-image

Deeploy version	1.38.x - 1.41.x	1.42.x - 1.48.x	Cloud & 1.49.0+
deeploy-cli	>= 1.38.0	>= 1.42.0	>= 1.49.0
kserve	0.12.0	0.13.1	0.15.1

Standard explainers

Tree SHAP

Deploy a tree SHAP explainer without training it yourself. Available only for tree-based classification models when using the XGBoost, Scikit-learn, or LightGBM model frameworks.

• Supported standard model frameworks: XGBoost, Scikit-learn and LightGBM
• Supported schema: v1
• Original prediction returned in /explain request: yes

Saliency

Deploy a saliency based explainer without training it yourself. Available only for text generation and text-to-text generation models when using the Hugging Face model framework.

• Supported standard model frameworks: Huggingface
• Supported schema: completion
• Original prediction returned in /completion with explain request: yes

Attention

Deploy an attention based explainer without training it yourself. Available only for text generation and text-to-text generation models when using the Hugging Face model framework.

• Supported standard model frameworks: Huggingface
• Supported schema: completion
• Original prediction returned in /completion with explain request: yes

Trained explainer frameworks

The versions described in the following sections are the recommended versions for the specific frameworks to use. If you deviate from the version, check for potential compatibility issues. Moreover, for each explainer (version) you can find whether the prediction is returned alongside when making request at the /explain. This can prevent doing redundant requests to obtain prediction separately.

SHAP

• Example: https://gitlab.com/deeploy-ml/sample-models/example-sklearn-census
• Expected pre-trained explainer artefact: dill (explainer.dill)

Deeploy version	<1.32.0	Cloud & 1.32.0+
shap	0.36.0	0.46.0
dill	0.3.3	0.3.7
original prediction returned	yes	yes

Anchors

• Example: https://gitlab.com/deeploy-ml/sample-models/income
• Expected pre-trained explainer artefact: dill (explainer.dill)

Deeploy version	<1.32.0	Cloud & 1.32.0+
alibi	0.6.4	0.9.4
dill	0.3.3	0.3.7
original prediction returned	no	no

MACE

• Example: https://gitlab.com/deeploy-ml/sample-models/example-fraud-detection
• Expected pre-trained explainer artefact: dill (explainer.dill)

Deeploy version	<1.32.0	Cloud & 1.32.0+
omnixai	1.1.4	1.3.1
dill	0.3.3	0.3.7
original prediction returned	no	no

PDP

• Expected pre-trained explainer artefact: dill (explainer.dill)

Deeploy version	<1.32.0	Cloud & 1.32.0+
omnixai	1.1.4	1.3.1
dill	0.3.3	0.3.7
original prediction returned	no	no

Custom explainer Docker images

Deeploy version	1.38.0 - 1.41.x	1.41.1.42.x - 1.48.x	Cloud & 1.49.0+
deeploy-cli	>= 1.38.0	>= 1.42.0	>= 1.49.0
kserve	0.12.0	0.13.1	0.15.1
original prediction returned	custom	custom	custom

Transformer frameworks

Custom transformer Docker images

Deeploy version	v1.38.0 - 1.41.x	v1.42.x - v1.48.x	Cloud & v1.49.0+
deeploy-cli	>= 1.38.0	>= 1.42.0	>= 1.49.0
kserve	0.12.0	0.13.1	0.15.1