Repository requirements

Before linking your repository to Deeploy, ensure it meets the following requirements:

1. Follows the repository contract
2. Contains a folder named 'model' that includes one of the following files:
   • model.<extension>; your machine learning model object.
   • reference.json; a reference to your machine learning model
3. (Optional) Contains a folder named 'explainer' that includes one of the following files:
   • explainer.dill; An explainer object created using the dill library.
   • reference.json; a reference to your explainer object or containerized image.
4. (Optional) Contains a folder named 'transformer' that includes following file:
   • reference.json; a reference to your transformer containerized image.

For more details on each requirement, consult the corresponding section of this article.

Beware

Repositories that don't adhere to the requirements will result in the failure of your Deployment

Repository contract

In order to successfully create a Deployment from a Repository, the git repository needs to adhere to a certain format. We call this format the contract.

repository
|__ model
|   |__ reference.json or model.<extension>
|
|__ explainer
|   |__ reference.json or explainer.<extension>
|
|__ transformer
    |__ reference.json

An example of the use of the contract, as seen in the Iris Proba example;

repository
|_ model
   |_ model.bst

...

It is possible to have multiple folders, each with their own contract to deploy multiple models from a single Repository.

Model folder

The model folder is mandatory and contains a (reference to a) machine learning model.

If you want to use a model file that holds your trained model, ensure compatibility and refer to the supported framework versions to determine the specific file type required for your chosen framework.

Caution

It is crucial that you have only one model file named model.extension (i.e. if you use my_model.<extension>, your Deployment will fail)

Alternatively, you can use the reference system

Explainer Folder

The explainer folder is optional and contains (a reference to) an explainer.

To facilitate saving and loading explainers, we use the dill library. For supported frameworks, refer to supported framework versions.

Caution

It is crucial that you have only one explainer file named explainer.dill (i.e. if you use my_explainer.dill, your Deployment will fail)

Transformer Folder

The transformer folder is optional and contains (a reference to) a transformer. Currently only containerized transformer image is supported. Object storage cannot be used for transformer.

Reference system and provenance

In situations where the size of individual files within a repository exceeds 100MB, or the total size surpasses 1GB, we strongly advise employing the referencing system. Using the reference system, the specific model is no longer stored in the git version history.

To uphold a consistent level of provenance, it is essential to upload a versioned new file to the Object Storage and update your reference.json whenever you intend to deploy a new version. Avoid overwriting files within Object Storage.

The reference system operates by replacing the contents of the ./model, ./explainer and ./transformer folders with a reference.json file, which serves as a reference to a folder stored in Object Storage or/and a containerized image (model/explainer/transformer). Deeploy currently offers support for AWS S3, Azure Blob Storage, and Google Cloud Storage (GCS).

Caution

It is crucial that you have only one reference file named reference.json (i.e. if you use my_reference.json, your Deployment will fail)

An example of the reference.json file, as seen in the Scikit-Learn census example;

{
    "reference": {
        "blob": {
            "url": "s3://deeploy-examples/sklearn/census/20220801171658/model"
        }
    }
}

When specifying an s3 bucket in a different region (Deeploy Cloud only) than eu-central-1 you need to specify it in the blob section.

# this reference is used in example repository: 
# https://gitlab.com/deeploy-ml/sample-models/example-pytorch-mnist
{  
 "reference": {
     "blob": {
         "url": "s3://deeploy-examples/pytorch/mnist/model",
         "region": "eu-west-1"
     }
  }
}

An example of the reference.json file that refers to a Docker image, as seen in the Custom Image Hello World Example

# this reference is used in example repository:
# https://gitlab.com/deeploy-ml/sample-models/example-custom-image-hello-world/
{
    "reference": {
        "docker": {
            "image": "deeployml/example-custom-image-hello-world:0.3.0",
            "port": 8080
        }
    }
}

Note

For model and explainer reference.json both docker and blob configurations can be specified.

{
    "reference": {
        "blob": {
            "url": "s3://path-to-blob"
        },
        "docker": {
            "image": "image-name",
            "port": port-number
        }
    }
}

An example of the reference.json that uses the Databricks Unity Catalog integration in combination with a model alias.

{
    "reference": {
        "databricks": {
            "model": "deeploy.default.model",
            "alias": "Champion"
        }
    }
}

An example of the reference.json that uses the Databricks Unity Catalog integration in combination with a model version.

{
    "reference": {
        "databricks": {
            "model": "deeploy.default.model",
            "version": "5"
        }
    }
}