'How to stop a "latent" Python Django cursor execute exception with SQLite
I am trying to automate a complete application schema rebuild in Django. Basically, drop all the old tables, delete the migration and cache file(s), delete the migration history, and then makemigrations and migrate. I understand this will have some scratching their heads but I like to configuration control my test data and load what I need from csv files and "starting from scratch" is much simpler with this model.
I'm hopeful the automation will work against all the different Django-supported databases so trying to come up with an approach that's generic (although some gaps and brute-force).
As noted above, the first step is to drop all the existing tables. Under the assumption that most model changes aren't deletes, I'm doing some class introspection to identify all the models and then attempting to discern if the related tables exist:
app_models = get_all_app_models(models.Model, app_label)
# Loop through and remove models that no longer exist by doing a count, which should
# work no matter the SQL DBMS.
with connection.cursor() as cursor:
for app_model in app_models[:]:
try:
cursor.execute (f'select count (*) from {app_model._meta.db_table}')
except OperationalError:
app_models.remove(app_model)
<lots more code, doing cursor and other stuff that works OK>
Once the above completes, app_models contains tables that remain and the code then works through dropping them (which itself isn't trivial in a generic way).
The processing is contained in a Django view/form and once complete it attempts to render a simple "okey dokey" page. The problem is that an exception is thrown during the render saying "no such table: xxx" and it refers to the execute statement. There is no other call stack context displayed.
The table mentioned was indeed referenced in the execute, in fact it was the last one in the original app_models. Somehow it seems that Django is retaining error information after the cursor is closed (the 'with' cleanup) and somehow executing against it during the render processing. I tried to execute a "good" SQL statement to "clear" the error, but no luck. I tried to "del" the cursor, also no luck.
Update 3/25: After many hours of trial and error, I have found that if I place the above code (along with the rest of the processing) in a separate function and call it from the view/form function I no longer get the spurious call-back to the execute statement. I researched SQLite quite a bit and it seems that OperationalError is handled differently than (say) DataError. I tried other approaches to clearing the error (e.g. closing the connection) without avail. I suspect the web server is doing something tricky with the stack and is treating the OperationalError incorrectly and the function nesting "hides" it. Cheers.
Sources
This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.
Source: Stack Overflow
| Solution | Source |
|---|
