Alerts
Alerts are useful to receive notifications when a metric has reached a defined threshold (e.g. when the response time of your model takes more than 100ms).
Triggered alerts
An overview of triggered alerts is available on the Alerts page. Use the alert status to let others know if an alert has been acknowledged or resolved.
Manage alert rules
Click Manage alert rules on the Alerts page.
Add alert rules
Click Add alert rule on the Manage alert rules page. Configure the following parameters:
- Name: displayed in notifications for this alert rule
- Metric: set up alert rules for any metric except disagreement per class and custom metrics.
- Severity: set to Warning by default. You can set up multiple alert rules for the same metric, each with a different severity level to reflect varying thresholds.
- Webhook: select a webhook to which data will be posted when the alert rule is triggered (optional).
The following parameters may be configurable depending on the selected metric:
- Feature: for feature-level metrics; select the feature to set up the alert.
- Threshold: the options for thresholds depend on the selected metric:
- Errors: the threshold can’t be customized because error-based alert rules automatically include detailed information about each error.
- Input validation: define a minimum and maximum threshold. If your Deployment metadata includes an observed minimum and maximum for the selected feature, they’ll appear here to help guide your selection.
- Other: define a minimum or a maximum threshold. If you want to have multiple thresholds, set up separate alert rules for each one.
- Cooldown: set a cooldown to avoid numerous triggered alerts in a short time. Select how long the alert should wait before triggering again after it’s been triggered once.
- Window: the alert rule is triggered based on the value of the metric in a (rolling) time window.
- Minimum sample size: set to 30 by default. Alerts won’t trigger if there are fewer predictions in the window.
This table includes some example alert rule configurations:
| Parameter | Example 1 | Example 2 |
|---|---|---|
| Name | Age of applicant | Performance |
| Metric | Input validation | Accuracy |
| Feature | Age | - |
| Severity | Warning | Critical |
| Threshold | Min. 18 - Max. 70 | Min. 80 |
| Cooldown | 12 hrs | 1 hr |
| Window | - | 1 hr |
| Min. Sample size | - | 100 |
Edit alert rules
Click the action button and select Edit. You can make changes to the elements described in the previous section.
Disable and enable alert rules
Click the action button and select Disable or Enable. Disabled alert rules remain visible in the overview (marked with an icon) but won’t trigger any alerts.
Delete alert rules
Click the action button and select Delete.
View alert rules in monitoring graphs
To view an alert rule in a monitoring graph, navigate to the graph and set the bin size to match the window of the alert rule.
Notifications
Deployment owners receive emails and in-product notifications for all alert rules in the Deployment by default. To change this, update your email and notification preferences on the Preferences page under your account.