Skip to main content
Version: Cloud

Creating Azure Machine Learning Deployments

With the Azure Machine Learning integration set up, you can create two types of managed Deployments: a Deeploy-hosted Deployment and an Azure Machine Learning-hosted Deployment. This guide explains both options and their differences.

Deeploy-hosted Deployments​

Deploy registered models and explainers from an Azure Machine Learning model registry to Deeploy's Deployment backend (KServe). This option provides access to all Deeploy features. During Deployment creation, Deeploy downloads the referenced model and explainer from your registry to Deeploy cloud storage and inserts them into their respective containers.

Prerequisites​

  • Add a model to the model registry in your Azure Machine Learning workspace.
  • (Optional) Add a Repository that adheres to the requirements. Repositories used for Azure Machine Learning Deployments use the reference system. If you don't specify a Repository, you can select "Generate a reference from Azure Machine Learning." This option retrieves all models and versions available in your Azure Machine Learning workspace. Deeploy ingests the selected model and version into the reference field.
  • Include an Azure Machine Learning (AzureML) reference, as shown in this example:
{
    "reference": {
        "azureML": {
            "model": "fraud_detection",
            "version": "1",
            "customPath: "model"
        }
    }
}

The model parameter is required. If you omit version, it defaults to 1. You can optionally set customPath to point to the folder where the model or explainer file is located. By default, the model path is set to the root artifact folder; for explainers, it's set to the explainer folder.

For an example implementation, see this repository.

You can use custom Docker containers with Azure Machine Learning models. Include the docker definition in your reference file:

{
    "reference": {
        "azureML": {
            "model": "fraud_detection",
            "version": "1",
            "customPath: "model"
        },
        "docker": {
            "image": "my-registry/my-image:1.0.0",
            "uri": "/v1/predict",
            "port": 8080
        }
    }
}

Authenticating to the Azure Machine Learning datastore​

Your Azure Machine Learning models are stored in one of your Workspace's configured datastores. Deeploy downloads the models from that datastore and must authenticate to it. Only Azure blob storage datastores are currently supported.

By default, Deeploy uses the Azure Machine Learning workspace's datastore credentials, which is either a storage account key or a Shared Access Signature (SAS) token. If these credentials are valid, no further action is required. If this authentication method is disabled in your Azure Machine Learning workspace, you can provide Azure blob storage credentials yourself. See blob credentials or temporary credentials.

Repository, branch, and commit​

Select a Repository, branch, and commit.

Deployment metadata​

Specify the name and retrieve metadata.

Model framework​

Specify the model framework. Custom Docker is supported.

Explainer framework​

Specify the explainer framework. Custom Docker is supported.

Transformer​

Configure Transformers. No specific Azure Machine Learning options are available.

Deploy​

Select Deploy. Deeploy initiates the automated deployment process.

Azure Machine Learning-hosted Deployments​

You can host your model in your own Azure Machine Learning environment. Some options aren't available with this hosting type, such as autoscaling or dynamic resource allocation. Each model or explainer is hosted on its own virtual machine in the Azure cloud.

Prerequisites​

  • A Docker container that can serve your model.
  • (Optional) Add a model to the model registry in your Azure Machine Learning workspace.
  • (Optional) Add a Repository that adheres to the requirements. Repositories used for Azure Machine Learning Deployments use the reference system.
  • Include an Azure Machine Learning (AzureML) reference, as shown in this example:
{
    "reference": {
        "azureML": {
            "image": "0c2df065d0f143c98e4b91f98cb91f10.azurecr.io/azure-machine-learning-fraud-detection-explainer:0.1.0",
            "uri": "/v1/models/fraud-detection:predict",
            "port": 8080,
            "readinessPath": "/v1/ready",
            "livenessPath": "/v1/health",
            "model": "fraud_detection",
            "version": "1"
        }
    }
}

The image, uri, port, readinessPath, and livenessPath parameters are mandatory. The model and version parameters are optional and refer to the name and version of a model in your Azure Machine Learning workspace's model registry.

If you specify model and version, the model files are available inside your model or explainer container at the path specified in the MODEL_BASE_PATH environment variable. The model name is available in the MODEL_NAME environment variable.

Your model and explainer images must have a readinessPath and livenessPath API route that returns a 200 status code when the container is running.

For an example implementation, see this repository.

Repository, branch, and commit​

Select a Repository, branch, and commit.

Deployment metadata​

Specify the name and retrieve metadata.

Model framework​

Only the Custom Docker model framework is allowed. You can't select a model framework in this step.

Select Advanced configuration to choose an instance type and instance count, which provides the appropriate mix of resources for your model. For guidance on choosing the instance type based on your use case, see the Azure virtual machine series guide.

Explainer framework​

Only the Custom Docker explainer framework is allowed. You can select No explainer or Custom docker in this step.

For Advanced configuration, you can select the instance type and count.

Transformer​

Transformers are disabled for Azure Machine Learning hosted Deployments. Include additional pre-processing and post-processing in the model or explainer Docker image.

Deploy​

Select Deploy. Deeploy initiates the automated deployment process.

Created resources​

Deeploy creates three Azure Machine Learning resources for your model and three for your explainer.

These resources include an Environment, an Online endpoint, and a Deployment.

You can find these resources in your Azure portal after the Deployment succeeds.