How to use trained model in Tensorflow Serving?

Question

I have trained Mask-RCNN model, and want to try Tensorflow Serving to use it in web.
So i just run tensorflow/serving docker container. And realised what i have no clue how to send image to predict. Here is saved_model_cli.py output:

signature_def['serving_default']:
  The given SavedModel SignatureDef contains the following input(s):
    inputs['input_anchors'] tensor_info:
        dtype: DT_FLOAT
        shape: (-1, -1, 4)
        name: input_anchors:0
    inputs['input_image'] tensor_info:
        dtype: DT_FLOAT
        shape: (-1, -1, -1, 6)
        name: input_image:0
    inputs['input_image_meta'] tensor_info:
        dtype: DT_FLOAT
        shape: (-1, 14)
        name: input_image_meta:0
  The given SavedModel SignatureDef contains the following output(s):
    outputs['mrcnn_detection/Reshape_1'] tensor_info:
        dtype: DT_FLOAT
        shape: (1, 100, 6)
        name: mrcnn_detection/Reshape_1:0
    outputs['mrcnn_mask/Reshape_1'] tensor_info:
        dtype: DT_FLOAT
        shape: (1, 100, 28, 28, 2)
        name: mrcnn_mask/Reshape_1:0
  Method name is: tensorflow/serving/predict

So i tried to send 6 bands image with Postman:

But get error. How can i form a correct request to serving?

Answer 1

You need to create a Json file with signature_name(Signature) and instances(images in list) like below,

data = json.dumps({"signature_name": "your_signature", "instances": test_data[0:n].tolist()})`

Then you can make a request like below,

headers = {"content-type": "application/json"}       
json_response = requests.post('http://localhost:8501/v1/models/your_model:predict', data=data, headers=headers)  
predictions = json.loads(json_response.text)['predictions']

You can refer this section in tfx tutorial to know more.