Skip to main content
Version: 1.39

V1

/predict

All model servers within a Deployment speak the Tensorflow V1 HTTP API.

The protocol looks as follows:

Request

{
"instances": [ <value>|<(nested)list>|<list-of-objects> ]
}

Instances: content of the input tensor

Response

{
"predictions": [ <value>|<(nested)list>|<list-of-objects> ]
}

Predictions: content of the output tensor

/Explain

All explainer servers within a Deployment speak the Tensorflow V1 HTTP API.

The protocol looks as follows:

Request

{
"instances": [ <value>|<(nested)list>|<list-of-objects> ]
}

Instances: content of the input tensor

Response

{
"explanations": [ <value>|<(nested)list>|<list-of-objects> ]
}

Predictions: content of the output tensor

V2

The V2 protocol increases utility and portability. Not all deployment frameworks support the V2 protocol, which is why, currently, all supported frameworks support only V1. When you create your own custom Docker, you are free to adopt V2.

The protocol looks as follows:

Request

{
"name": $string,
"shape": [ $number, ... ],
"datatype": $string,
"parameters": $parameters,
"data": [ <value>|<(nested)list>|<list-of-objects> ]
}
  • name: name of the input tensor
  • shape: shape of the input tensor. Each dimension is an integer
  • datatype: datatype of tensor input elements as defined here
  • parameters (optional): object containing 0 or more parameters - as explained here
  • data: content of the input tensor. More information can be found here

Response

{
"model_name": $string,
"model_version": $string,
"id": $string,
"parameters": $parameters,
"outputs": [$response_output, ... ]
}
  • name: name of the input tensor
  • id: the identifier given to the request
  • parameters (optional): object containing 0 or more parameters as explained here
  • outputs: see response_output below.

Response_output

{
"name": $string
"shape": [$number, ... ],
"datatype": $string,
"data": [$tensor_data, ... ]
}
  • name: name of the output tensor
  • shape: shape of the input tensor. Each dimension is an integer
  • datatype: datatype of tensor output elements as defined here
  • data: content of the output tensor. More information can be found here