'awswrangler flatten function: No such struct field element
I have a highly nested data frame and am getting the following error:
An error was encountered:
No such struct field element in eqp_item_type_txt, eqp_msrp_amt, eqp_make_nm, eqp_ser_num, eqp_mfg_yr, camp_unit_cmp_ftrm_prmamt, camp_unit_col_ftrm_prmamt; line 1 pos 0
using the following flatten function. Any reason why I am getting this? The schema of the dataframe is like this:
|-- a: array (nullable = true)
| |-- element: struct (containsNull = true)
| | |-- b: string (nullable = true)
| | |-- c: string (nullable = true)
| | |-- d: array (nullable = true)
| | | |-- element: struct (containsNull = true)
| | | | |-- d1: string (nullable = true)
| | | | |-- d2: string (nullable = true)
Would I have to explode out the arrays first before using the flatten function?
Sources
This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.
Source: Stack Overflow
| Solution | Source |
|---|
