'How to run multiple queries as a transaction in sqlalchemy and capture errors without stopping on failures

I have the following table in Postgres:

t_product

product_id    value
==========    =====
   A           1.2
   B           1.3
   C           2.1

(there is a NOT NULL constraint on the 'value' column. And 'product_id' is the primary key)

I plan on doing multiple upserts in a single transaction to the above t_product table using a raw query on sqlalchemy. In case one of the upsert statements violate the NOT NULL constraint of the value column (or violates any other constraint), I don't want all the other queries to fail, but i do need to be notified about the failures. (The reason I am running the queries in a single transaction is to improve the speed of my code since a lot of upserts are going to have to be done)

For example:

product_values = [['A', 12.4], ['C','NULL'], ['D',1.1]]

# database_engine is the sqlalchemy engine object
with database_engine.connect() as connection:
    with connection.begin() as transaction:
        for product_value in product_values:
                query = """INSERT INTO t_product (product_id, value)
                           VALUES(product_value[0],str(product_value[1])
                           ON CONFLICT (product_id)
                           DO 
                           UPDATE SET value = EXCLUDED.value"""
                _ = connection.execute(query)

On running the above query, I want to be able to capture the fact that the product C was not able to be upserted since it violated the NOT NULL constraint of the value column. But at the same time I want all the remaining products (i.e. A,D) to be upserted successfully.

How do I modify my approach to allow this to happen?



Sources

This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.

Source: Stack Overflow

Solution Source