'How to use Tensorflow Serving restful API on multiple named inputs estimator model?

Tensorflow version: 1.14.0

Tensorflow serving version: 1.14.0


Description

The saved estimator model has multiple named inputs, whose keys are input_1, input_2... And these inputs are tf.feature_column.*, I saved serving model using codes below, where feature_columns is a list of inputs:

estimator.train(input_fn=lambda: train_input_fn(features, 1000), steps=100)
serving_input_fn = tf.estimator.export.build_parsing_serving_input_receiver_fn(
        tf.feature_column.make_parse_example_spec(feature_columns))
estimator.export_saved_model("from_estimator/", serving_input_fn)

After model saved, I use saved_model_cli show the inputs and outputs of this model:

saved_model_cli show --dir /from_estimator/1572335377/ --tag_set serve --signature_def serving_default

which print:

The given SavedModel SignatureDef contains the following input(s):
  inputs['examples'] tensor_info:
      dtype: DT_STRING
      shape: (-1)
      name: input_example_tensor:0
The given SavedModel SignatureDef contains the following output(s):
  outputs['logits'] tensor_info:
      dtype: DT_FLOAT
      shape: (-1, 100)
      name: MatMul_1:0
  outputs['probabilities'] tensor_info:
      dtype: DT_FLOAT
      shape: (-1, 100)
      name: Softmax:0
Method name is: tensorflow/serving/predict

Note that the inputs are not the expected keys input_1 and input_2, but examples. Confusing about it !!

Whatever, I using saved_model_cli run the model locally by:

saved_model_cli run --dir from_estimator/1572335377/ --tag_set serve --signature_def serving_default --input_examples "examples=[{'input_1': [1],'input_2': [1]}]"

And it gave the right outputs.

Question

I deployed the model using docker image of tensorflow serving, and tried to get the outputs from restful api. I've tried the request format as follow, but all of them failed:

  1. curl -d '{"instances": [{"input_1": [1],"input_2": [1]}]}' -X POST http://localhost:8501/v1/models/from_estimator:predict, it gave the wrong key of input error.
  2. curl -d '{"instances": [{"examples": {"input_1": [1],"input_2": [1]}}]}' -X POST http://localhost:8501/v1/models/from_estimator:predict, it gave Invalid argument: JSON Value: not formatted correctly for base64 data error.
  3. curl -d '{"instances": [{"examples": "{\"input_1\": [1],\"input_2\": [1]}"}]}' -X POST http://localhost:8501/v1/models/from_estimator:predict, it gave couldn't parse error.

So how to set the multiple named inputs properly? thanks~



Sources

This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.

Source: Stack Overflow

Solution Source