Python code to execute multiple Snowflake queries in parallel

Question

I have written a python program to fetch queries from an excel and print the output in an CSV format. The code is working fine if i run the queries sequentially one by one but of I try to use execute_async to run all the queries parallely then my code isn't working at all

Could somebody please help me out here. Below is my python code

            from SNOWFLAKE_CONNECTION import *
            import pandas
            
            wb = openpyxl.load_workbook(r"Query_Sheet.xlsx")
            ws = wb.get_sheet_by_name('Sheet1')
            all_rows = list(ws.rows)
            cur = ctx.cursor()
            # Pull information from specific cells.
            for row in all_rows[1:4]:
                scenario = row[1].value
                query = row[2].value
                if_execute = row[3].value
                if if_execute == 'Y':
                    try:
                        cur.execute_async(query)
                        df = cur.fetch_pandas_all()
                    except:
                        print(scenario," Failed")
                    else:
                        print(df)

All i'm getting here is Scenario failed as the output

Answer 1

Does the order of results matter? If not then you can use execute_async in this way:

# Submit an asynchronous query for execution.
cur.execute_async(query)

# Retrieve the results.
cur.get_results_from_sfqid(query_id)
results = cur.fetchall()
print(f'{results[0]}')

See more examples here