Evaluate

This section explains how to evaluate predictions of a Deployment.

Prediction Evaluation

Evaluating a prediction means overwriting, or correcting, the model's output. This can be used as a reference, or to be used as training data for a next iteration of the model.

Predictions are evaluated using a prediction log's ID. These can be retrieved using the prediction log methods.

from deeploy import CreateEvaluation

evaluation_input: CreateEvaluation = {
    "agree": False,
    "desired_output": { "predictions": [True] },
    "comment": "Example evaluation from the Python Client",
}

evaluation = client.evaluate(deployment_id, prediction_log_id, evaluation_input)