Category "google-bigquery"

BigQuery DENSE_RANK

I posted something similar to this but without the full code and I don't think people understood what I was looking for... I'm writing a query to examine bus st

Get Syntax Error: Expected end of input but got keyword ORDER at [5:1]: in BigQuery

When I run the following I get the error Syntax Error: Expected end of input but got keyword ORDER at [5:1]: SELECT purchase_price FROM test-project-349410.cust

Apache Beam with Dataflow: flag 'ignore_unknown_columns' for WriteToBigQuery not working

I am building a streaming pipeline using Apache Beam (Python SDK version 2.37.0) and Google Dataflow to write some data I receive via Pubsub to BigQuery. I proc

SQL: how to check if at least one keyword from a keyword list exists in a string

I have a table with the following schema, id data_string 1 I have a pen. 2 Monday is not good. 3 I love Friday. ... ... And I also have a list of k

Count overnight hours as one day

I have a dataset where certain operations occur during the overnight hours which I'd like to attribute to the day before. For example, anything happening betwee

What is the formula in SQL to calculate time/ number of days taken to sell x units of a product since its launch?

As part of a dashboard development, I need to know the number of days taken by a newly launched item to reach sales of 1000 units. This is for benchmarking perf

Querying JSON value in BigQuery

I have a BigQuery table ("activity") where one column ("groups") contains a JSON value (the table is an export of Google Workspace logs). I'd like to be able to

SCD2 on nested table in GCP big query

I am new to gcp big query. I am trying to model scd2 implementation on both outer and nested table on GCP big query. I would like to know your suggestions wethe

Can't SET variable in BigQuery | Error: Cannot coerce expression

I'm trying to set variable in order to make further operation. But when I set my first variable on line 2 ("SET(new_user)=( "), I get an error Query error: Can

How can I create an Entity Relationship diagram (ER) For BigQuery Tables/ Views [closed]

I have some complex BigQuery views that nest each other. I would like to map them in an ER diagram so they are easier to visualize. I can do t

Upload to BigQuery from Cloud Storage

Have ~50k compressed (gzip) json files daily that need to be uploaded to BQ with some transformation, no API calls. The size of the files may be up to 1Gb. What

GCP Bigquery - Time Travel - Max and Min Window time for a table

I understand the BigQuery provides 7 days time travel. I'm trying to get Max and Min of time travel possible for each table like Min and Max of timestamp. Is th

BigQuery table creation from google cloud storage

I am trying to create a table in Google BigQuery Table in the console. I have given the correct GCS bucket address where the CSV resides. The CSV is about 140GB

get Forbidden: 403 Access Denied when do request to transfer data from google cloud storage to bigquery using python

I'm owner in google cloud console. I want transfer data from gcs to Bigquery, but I get "Forbidden: 403 Access Denied: File gs://data/silver/rates/nbu_rates/nbu

Clickstream performance Scoring formula using SQL

imagine we have a clickstream and we want to have some everyday score that follows next rules: increase by 1 each impression increase by 10 each click decrease

How do I create a table with multiple auto incrementing values?

I would like to have an autoincrementing string in my BigQuery table. This seems like a good way of doing it: CREATE TABLE dbo.YourTable ( ID INT IDENTITY(1

Bigquery - which table are affected by another table

I want to delete a table in bigquery. But before that, I want to know which tables are being linked to the table I'm about to delete. Is there a query to extrac

Create/Query different tables based on conditional statement in BQ SQL

To illustrate with an example, say I have two tables in BigQuery called "fruits" and "vegetables" respectively. I want to have a variable at the start that lets

Defining Month Time Partition using Teradata Data Transfer Service Custom Schema

I am looking for a way to define a Teradata Data Transfer custom schema that implements a month based date partition. The documentation only provides a method

Can DBT directly connect to Google sheets?

we currently have a DBT instance that sits over our Google BigQuery data warehouse. Now we've recently been asked to incorporate some data from Google Sheets in