I am building a data pipeline from PubSub to Beam (Direct/Dataflow Runner) to Big Query. Today we started to run into issues where beam IO BigQuery connector st
I would like to repeat a block of code with a complex operation (defined in a subquery) for many columns in bigquery. I cannot find a better way than copy and p
I have this table 1 in Bigquery and I need to count the elements in column segments and category that correspond to a single user id. Desired outcome presented
Running this query on BigQuery but resulting in this error: Error: Scalar subquery produced more than one element select * from CRM.organizations where name L
I am trying to setup a bigquery data transfer configuration using terraform. I am using my personal gcp account. I have a setup of terraform on my laptop so tha
I'm simply trying to get one data from bigquery from python. Below is the query select max(date) from `x.x.table_name` all the requirements to access bigquery
Is there any way to write unstructured data to a big query table using apache beam dataflow big query io API (i.e without providing schema upfront)
Having an issue with Scheduled Queries, using a python script, I get the error: "google.api_core.exceptions.PermissionDenied: 403 The caller does not have permi
I'm trying to pull product search colors from urls in my bigquery data, but the urls format changed at one point so I'm trying to pull them from two different f
I have a notebook in which I access data through APIs, work with the data and send the results to a BigQuery table as well as to a GCS bucket. Everything works
I'm trying to create a BigQuery snapshot of a table with an expiration date 60 minutes in the future. It looks like I can't define a expiration time when creati
I'm planning a pipeline from an S3 bucket to BigQuery. I have set up daily bigquery data transfers from S3 as described here. So, I have a data transfer configu
I started with a date in a string format from a JSON extraction using this: json_value(answer, '$.date_created') and got an output 2020-01-02T10:26:47.056-04:00
I have a table in BigTable, with a single column family, containing some lead data. I was following the Google Cloud guide to querying BigTable data from BigTab
I'm new to terraform. Is it possible to load the content of a CSV file into a BigQuery table without uploading it to GCS? I've studied the document below, but t
So there is a requirement to build tables and charts in google sheets with data in bigquery. The data can be brought in to google sheets using connected sheets.
Let's take the following example from BigQuery, but it's the same across all other databases I've seen: with tbl as ( select [1,2,3] as arr, '{"IDS": [1
I integrated my Firestore solution with BigQuery. With every Firestore Insert/Update/Delete operation the Data gets transferred to BigQuery. I am trying to sche
Getting below error while connecting Simba driver with BigQuery on OBIEE UNIX Env , any suggesting's. Do we have some license file associated with the driver. U
I am following this guide in setting up a vertex AI pipeline. I am told to supply the "BigQuery project for exporting data", except that the guide is not clear