'How to use Tensorflow Serving restful API on multiple named inputs estimator model?
Tensorflow version: 1.14.0
Tensorflow serving version: 1.14.0
Description
The saved estimator model has multiple named inputs, whose keys are input_1, input_2... And these inputs are tf.feature_column.*, I saved serving model using codes below, where feature_columns is a list of inputs:
estimator.train(input_fn=lambda: train_input_fn(features, 1000), steps=100)
serving_input_fn = tf.estimator.export.build_parsing_serving_input_receiver_fn(
tf.feature_column.make_parse_example_spec(feature_columns))
estimator.export_saved_model("from_estimator/", serving_input_fn)
After model saved, I use saved_model_cli show the inputs and outputs of this model:
saved_model_cli show --dir /from_estimator/1572335377/ --tag_set serve --signature_def serving_default
which print:
The given SavedModel SignatureDef contains the following input(s):
inputs['examples'] tensor_info:
dtype: DT_STRING
shape: (-1)
name: input_example_tensor:0
The given SavedModel SignatureDef contains the following output(s):
outputs['logits'] tensor_info:
dtype: DT_FLOAT
shape: (-1, 100)
name: MatMul_1:0
outputs['probabilities'] tensor_info:
dtype: DT_FLOAT
shape: (-1, 100)
name: Softmax:0
Method name is: tensorflow/serving/predict
Note that the inputs are not the expected keys input_1 and input_2, but examples. Confusing about it !!
Whatever, I using saved_model_cli run the model locally by:
saved_model_cli run --dir from_estimator/1572335377/ --tag_set serve --signature_def serving_default --input_examples "examples=[{'input_1': [1],'input_2': [1]}]"
And it gave the right outputs.
Question
I deployed the model using docker image of tensorflow serving, and tried to get the outputs from restful api. I've tried the request format as follow, but all of them failed:
curl -d '{"instances": [{"input_1": [1],"input_2": [1]}]}' -X POST http://localhost:8501/v1/models/from_estimator:predict, it gave thewrong key of inputerror.curl -d '{"instances": [{"examples": {"input_1": [1],"input_2": [1]}}]}' -X POST http://localhost:8501/v1/models/from_estimator:predict, it gaveInvalid argument: JSON Value: not formatted correctly for base64 dataerror.curl -d '{"instances": [{"examples": "{\"input_1\": [1],\"input_2\": [1]}"}]}' -X POST http://localhost:8501/v1/models/from_estimator:predict, it gavecouldn't parseerror.
So how to set the multiple named inputs properly? thanks~
Sources
This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.
Source: Stack Overflow
| Solution | Source |
|---|
