Flask POST request optimization

Question

I have a flask app running on my machine, and I'm sending a post request to it from a thread on my own machine. The flask app then sends it to disk, trains a model on it, and pushes the data out (publish). I'm timing each step because it seems slow. Turns out the slowest part is getting the data from my csv (called spoof).

Module	end	UTC now	timecost
SPOOF	IN	...35:00.110285	0
SPOOF	OUT	...35:00.112283	0.001998
FLASK	IN	...35:02.160254	2.047971
FLASK	OUT	...35:02.167264	0.007010
DATA	IN	...35:02.168251	..
DATA DISK	IN	...35:02.168251	..
DATA DISK	OUT	...35:02.176605	..
DATA	OUT	...35:02.179534	..
MODEL	IN	...35:02.180788	..
MODEL	OUT	...35:02.495012	0.314224
PUBLISH	IN	...35:02.496034	..
PUBLISH	OUT	...35:02.496534	..

getting it to flask takes 2 seconds while training the model on new data takes less than a third of a second. I must be doing something very wrong.

Here's the thread that sends the post request:

def run(self):
    while True:
        print('SPOOF IN', dt.datetime.utcnow())
        x = self.provideIncrementalWithId()
        print('SPOOF OUT', dt.datetime.utcnow())
        response = requests.post(
            url=f'http://localhost:{self.port}/subscription/update', 
            json=x)

here's basically the entire flask app but I think only update is relevant:

import threading
import secrets
import satori
import datetime as dt
from flask import Flask
from flask import request
from waitress import serve
from satori.lib.engine.structs import Observation

app = Flask(__name__)
app.config['SECRET_KEY'] = secrets.token_urlsafe(16)
Engine = satori.getEngine()
Engine.run()

def spoofStreamer():
    thread = threading.Thread(target=satori.spoof.Streamr().run, daemon=True)
    thread.start()

@app.route('/subscription/update', methods=['POST'])
def update():
    print('FLASK IN', dt.datetime.utcnow())
    x = Observation(request.json)
    print('FLASK OUT', dt.datetime.utcnow())
    Engine.data.newData.on_next(x)
    return request.json

if __name__ == '__main__':
    spoofStreamer()
    serve(app, host='0.0.0.0', port=satori.config.get()['port'])

Is there something stupid I'm doing here? Should I not use waitress? Why is it so slow, especially given the request is coming from the same machine?

EDIT: I just realized maybe it's a versioning thing here are my relevant versions:

package	version
Python	3.9.5
Flask	2.0.1
click	8.0.0rc1
colorama	0.4.4
itsdangerous	2.0.1
Jinja2	3.0.1
MarkupSafe	2.0.1
Werkzeug	2.0.1
waitress	2.0.0

Edit2: after updating my packages to the latest I still get 2 seconds to get the request

Module	end	UTC now	timecost
SPOOF	IN	..00:24.821877	0
SPOOF	OUT	..00:24.823876	0.001999
FLASK	IN	..00:26.873730	2.049854
FLASK	OUT	..00:26.880726	0.006996

Answer 1

You know what it was... I changed localhost to 127.0.0.1 in the post call and its almost instant now.