'Snapshot fails because of "Resources exceeded during query execution"

At the end of my dbt pipeline that transform my data in a bigquery database I have scheduled some basic snapshots. This snap enables me to picture a calendar table with sales forecast for each day. It's quite basic and not complexe.

Never had any issue with all my other snap before but with that one I have the following error :

"Database Error in snapshot sales_forecasts_snapshot (snapshots/sales_forecasts_snapshot.sql)
07:55:51    Resources exceeded during query execution: Not enough resources for query planning - too many subqueries or query is too complex."

Here is the code :

'''{% snapshot sales_forecasts_snapshot %}
{{
config(
  target_database='baobab-304613',
  target_schema='snapshots',
  unique_key='calendar_day',
  strategy='timestamp',
  updated_at='updated_at',
)
}}

select
 calendar_day                                                   AS calendar_day
,CAST(total_forecast_sales_paying_members AS NUMERIC)           AS total_forecast_sales_paying_members
,CAST(forecast_nb_paying_members AS NUMERIC)                    AS forecast_nb_paying_members
,CAST(forecast_sales_per_paying_member AS NUMERIC)              AS forecast_sales_per_paying_member

,CAST(total_forecast_sales_free_members AS NUMERIC)             AS total_forecast_sales_free_members
,CAST(forecast_nb_free_members AS NUMERIC)                      AS forecast_nb_free_members
,CAST(forecast_sales_per_free_member AS NUMERIC)                AS forecast_sales_per_free_member

,CAST(total_forecast_sales_free_trialer AS NUMERIC)             AS total_forecast_sales_free_trialer
,CAST(forecast_nb_free_trialers AS NUMERIC)                     AS forecast_nb_free_trialers
,CAST(forecast_sales_per_free_trialer AS NUMERIC)               AS forecast_sales_per_free_trialer

,CAST(total_forecast_sales_active_customer_90_days AS NUMERIC)  AS total_forecast_sales_active_customer_90_days
,CAST(forecast_sales_per_active_customer_90_days AS NUMERIC)    AS forecast_sales_per_active_customer_90_days
,CAST(forecast_nb_active_customers_90_days AS NUMERIC)          AS forecast_nb_active_customers_90_days

,updated_at                                                     AS updated_at --this date is the current_date (the date of the day --> the idea is to capture every day the new forecasts) 

from {{ ref('sales_forecasts') }}

{% endsnapshot %}'''

This snapshot is based on a view which runs without any issue. This snapshot is performed with dbt in a bigquery database. When I create the snapshot, it runs perfectly well the first time it runs but then fail at the second launch. What surprises me is also that even if my pipeline fails because of that, it seems like it still run as I have fresh data in my database.



Sources

This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.

Source: Stack Overflow

Solution Source