WebSocket not working when trying to send generated answer by keras

Question

I am implementing a simple chatbot using keras and WebSockets. I now have a model that can make a prediction about the user input and send the according answer.

When I do it through command line it works fine, however when I try to send the answer through my WebSocket, the WebSocket doesn't even start anymore.

Here is my working WebSocket code:

@sock.route('/api')
def echo(sock):
    while True:
        # get user input from browser
        user_input = sock.receive()
        # print user input on console
        print(user_input)
        # read answer from console
        response = input()
        # send response to browser
        sock.send(response)

Here is my code to communicate with the keras model on command line:

while True:
    question = input("")
    ints = predict(question)
    answer = response(ints, json_data)
    print(answer)

Used methods are those:

def predict(sentence):
    bag_of_words = convert_sentence_in_bag_of_words(sentence)
    # pass bag as list and get index 0
    prediction = model.predict(np.array([bag_of_words]))[0]
    ERROR_THRESHOLD = 0.25
    accepted_results = [[tag, probability] for tag, probability in enumerate(prediction) if probability > ERROR_THRESHOLD]

    accepted_results.sort(key=lambda x: x[1], reverse=True)

    output = []
    for accepted_result in accepted_results:
        output.append({'intent': classes[accepted_result[0]], 'probability': str(accepted_result[1])})
        print(output)
    return output


def response(intents, json):
    tag = intents[0]['intent']
    intents_as_list = json['intents']
    for i in intents_as_list:
        if i['tag'] == tag:
            res = random.choice(i['responses'])
            break
    return res

So when I start the WebSocket with the working code I get this output:

 * Running on http://127.0.0.1:5000/ (Press CTRL+C to quit)
 * Restarting with stat
 * Serving Flask app 'server' (lazy loading)
 * Environment: production
   WARNING: This is a development server. Do not use it in a production deployment.
   Use a production WSGI server instead.
 * Debug mode: on

But as soon as I have anything of my model in the server.py class I get this output:

2022-02-13 11:31:38.887640: I tensorflow/core/common_runtime/pluggable_device/pluggable_device_factory.cc:305] Could not identify NUMA node of platform GPU ID 0, defaulting to 0. Your kernel may not have been built with NUMA support.
2022-02-13 11:31:38.887734: I tensorflow/core/common_runtime/pluggable_device/pluggable_device_factory.cc:271] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 0 MB memory) -> physical PluggableDevice (device: 0, name: METAL, pci bus id: <undefined>)
Metal device set to: Apple M1

systemMemory: 16.00 GB
maxCacheSize: 5.33 GB

It is enough when I just have an import at the top like this: from chatty import response, predict - even though they are unused.

Answer 1

There is no problem with your websocket route. Could you please share how you are triggering this route? Websocket is a different protocol and I'm suspecting that you are using a HTTP client to test websocket. For example in Postman:

Postman New Screen

HTTP requests are different than websocket requests. So, you should use appropriate client to test websocket.