'Python code with execute_async is not fetching the query results asynchronously
I had asked a question previously in the SO and based on the guidance from that post I drafted a code to fetch the queries from an excel and asynchronously run and print the results so that the queries that runs faster is displayed first and the late running ones are displayed last. the code I have written fetches the queries form excel and sequentially prints the result still which is not the one I was expecting.
The input sheet is kept it in a way that large query is placed in the first row and smaller ones are placed in the subsequent rows.
Could somebody please help me out here where it is going wrong
from SNOWFLAKE_CONNECTION import *
import pandas
import time
df = pd.read_excel(r"input_path\INPUT_SHEET.xlsx","Sheet1",engine='openpyxl')
queries = len(df)
print(str(queries) + 'Queries')
query = []
complete = []
for item in range(queries):
query.append(item)
complete.append(0)
cs=ctx.cursor()
process_complete = 0
process_pass = 0
for qry in query:
if df.iloc[qry][3]=='Y':
sql = df.iloc[qry][2]
#scenario = df.iloc[qry][1]
try:
cs.execute_async(sql)
except:
Print('Skipped a sceanrio')
else:
query[qry]=cs.sfqid
while process_complete == 0:
item = -1
process_pass += 1
if sum(complete) == queries or process_pass == 10:
process_complete = 1
for result in query:
item += 1
if complete[item] == 0:
print('result for:' +str(result))
status = ctx.get_query_status(result)
if str(status) == 'QueryStatus.SUCCESS':
complete[item] = 1
cs.get_results_from_sfqid(result)
recs = cs.fetchall()
print('query' + str(item))
else :
time.sleep(1)
ctx.close()
print('done')
Sources
This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.
Source: Stack Overflow
| Solution | Source |
|---|
