'How to keep a Postgresql database connection open with Python

I am running a loop, and in each iteration I go to the PostGres database to find certain matching records. My issue is that it takes too long to open a new database connection at each time through the loop.

import psycopg2
from psycopg2 import sql
from sqlalchemy import create_engine

engine = create_engine(mycredentials)
db_connect = engine.connect()

query = ("""SELECT ("CustomerId") AS "CustomerId", 
                ("LastName") AS "LastName", 
                ("FirstName") AS "FirstName",
                ("City") AS "City", 
                FROM 
                    public."postgres_table" 
                WHERE (LEFT("LastName", 4) = %(blocking_lastname)s 
                    AND LEFT("City", 4) = %(blocking_city)s) 
                    OR (LEFT("FirstName", 3) = %(blocking_firstname)s
                    AND LEFT("City", 4) = %(blocking_city)s)
""")

for index, row in df_loop.iterrows():
    blocking_lastname = row['LastName'][:4]  
    blocking_firstname = row['FirstName'][:3]  
    blocking_city = row['City'][:4]  

    df = pd.read_sql(query, db_connect, params= {
                                        'blocking_firstname': blocking_firstname,
                                        'blocking_lastname': blocking_lastname, 
                                        'blocking_city': blocking_city})

This code works, but it doesn't appear to be keeping the database connection open since there is too much latency with the query. (Note: The table columns have indexes.)

UPDATE: I updated the code above, using db_connect as an object outside of the loop rather than engine.connect() as a call in pd.read_sql() as I originally had it written.



Sources

This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.

Source: Stack Overflow

Solution Source