Skip to main content
Version: 1.36

Serverless Deployments

There are two ways to deploy your Deployments:

  • non-serverless
  • serverless

This article explores the differences and similarities between these two options, helping you make an informed decision about which option is best suited for your needs.

Similarities and differences​

Both non-serverless and serverless Deployment methods offer automatic scaling to handle incoming traffic. This means that as the usage of your model increases, additional instances of your model will be deployed to accommodate the growing traffic. Conversely, if there are more instances than required to serve the incoming traffic, your deployment will scale down accordingly.

While there are similarities, unique elements to both non-serverless and serverless Deployment.

Non-serverless​

Non-serverless deployments are particularly well-suited for real-time applications. Since they are always active and consistently utilize resources, these deployments can readily respond to incoming requests without delay.

If your application falls into the category of real-time applications, opting for a non-serverless deployment is recommended.

Serverless​

Serverless deployments operate differently by not continuously consuming resources. Instead, they activate when the first request is received and shut down completely if there is no further activity for a period of time.

This approach is cost-effective when your deployment is not actively used for fetching predictions. However, it's important to be aware of a potential drawback known as a "cold-start." This refers to the delay in serving the first request as the application needs to boot up.

If your application does not require real-time processing, such as running once a day, choosing a serverless deployment option is recommended.

What should you choose?​

If your applications is a real-time application, choose the non-serverless deployment. If your application is not real-time (e.g. runs once a day), go with the serverless option.

If, for whatever reason you're not sure, we suggest the non-serverless option.