'how to append +30 tables together using sql bigquery if each table has a different number of columns

I have 30+ tables, all with different #'s of columns.

I'm hoping to append all the tables together in Google BigQuery using SQL into 1 single table that would share the same column names

i'm having trouble with the query below. I've essentially selected the desired column names i want (but not all columns appear in each table)

SELECT record, uuid, date, status, q1r1, q1r2, q1r3, q1r4, q1r5, q1r6, q1r7, q1r99, q2, q3, q4, q5, q6a, q6b, q7a, q7b, q8a, q8b, q9, q10, from Table 1
UNION ALL 
SELECT record, uuid, date, status, q1r1, q1r2, q1r3, q1r4, q1r5, q1r6, q1r7, q1r99, q2, q3, q4, q5, q6a, q6b, q7a, q7b, q8a, q8b, q9, q10, from Table 2
UNION ALL 
select record, uuid, date, status, q1r1, q1r2, q1r3, q1r4, q1r5, q1r6, q1r7, q1r99, q2, q3, q4, q5, q6a, q6b, q7a, q7b, q8a, q8b, q9, q10, from Table 3

i'm getting the error "Unrecognized name: q6b; Did you mean q6a? at [1:194]"

I suspect where i went wrong is that i selected column names (ex: g6b) that are not in the table.

instead of getting the column names for each 30+ of my datasets and using those in the select query, can I just use some type of null/0 value for the tables that don't have the columns?

For example, if Table 1 has q1, q3, q6b and table 2 has q1, q2, q3, q6a, can i do this as my query

SELECT record, uuid, q1, q3, q6b, null, null from Table 1
UNION ALL
Select record, uuid, q1, 2, q3, q6a, null from Table 2

With the above, I get the error "UNION ALL has incompatible types: STRING, NUMERIC" the record column along with a few others has numerical values while others have text. Is there an easy way to update this? I have ~250 columns in each table and the ones that have numbers vs. text are scatterd throughout so it's going to take time on my end to determine which ones have numbers and which ones have text.

I'm quite new to sql & never done unions before, so any help is appreciated!



Sources

This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.

Source: Stack Overflow

Solution Source