'Request TorchServe service inside a containerized app
I'm creating a text generation app, and I want to use TorchServe to make it scale well
In my DockerFile, I first run a CMD
to run TorchServe, then run the app with ENTYPOINT
...
CMD ["torchserve", "--start", "--model-store", "/model_store", "--ts-config", "/config.properties"]
ENTRYPOINT ["python3", "-X", "utf8", "app.py"]
And in my app.py I request the local TorchServe url to get my predictions
TORCH_URL = "http://127.0.0.1:8989/predictions/"
def request_torch(artist_name, input_text):
"""
Call torchserve service to get inference
"""
output = requests.post(TORCH_URL+ artist_name,
data = {'body':input_text})
decoded_output = output.content.decode('utf-8')
return decoded_output
Problem is : when deploying my app on gcloud, I get a "connection refused" error
NewConnectionError('<urllib3.connection.HTTPConnection object at 0x3e6cfe0cb7b8>: Failed to establish a new connection: [Errno 111] Connection refused'
Is it possible to call the TorchServe service locally inside such an app ?
Sources
This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.
Source: Stack Overflow
Solution | Source |
---|