'Convert a JSON export file to BQ JSON Newline delimited JSON

I have a JSON export from a database and I'd like to upload it and create a new table on BQ. This file is 600MB and I tried to use the jq on mac terminal but I'm a noob and I couldn't do it...

Is there any way to a convert a random json file and get the result into this newline delimited JSON file? If yes, pls help me with this

json google-bigquery

Solution 1:^[1]

You can load your JSON into cloud storage following this documentation.

Also with this code (python), you can load into BigQuery previously stored in a bucket.

from google.cloud import bigquery

# Construct a BigQuery client object.
client = bigquery.Client()

# TODO(developer): Set table_id to the ID of the table to create.
# table_id = "your-project.your_dataset.your_table_name"

job_config = bigquery.LoadJobConfig(
    schema=[
        bigquery.SchemaField("name", "STRING"),
        bigquery.SchemaField("post_abbr", "STRING"),
    ],
    source_format=bigquery.SourceFormat.NEWLINE_DELIMITED_JSON,
)
uri = "gs://cloud-samples-data/bigquery/us-states/us-states.json"

load_job = client.load_table_from_uri(
    uri,
    table_id,
    location="US",  # Must match the destination dataset location.
    job_config=job_config,
)  # Make an API request.

load_job.result()  # Waits for the job to complete.

destination_table = client.get_table(table_id)
print("Loaded {} rows.".format(destination_table.num_rows))

Sources

This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.

Source: Stack Overflow

Solution	Source
Solution 1	Eduardo Ortiz

'Convert a JSON export file to BQ JSON Newline delimited JSON

Solution 1:[1]

Sources

Related Questions

Solution 1:^[1]