'How do I restructure a json in YAML?

I would like to send data from an API to a BigQuery table with Google Workflows (YAML format). But the API response that I want to send to BigQuery table does not match the "insertAll" BigQuery connector feature.

main:
   params: [input]
   steps:
   - retrieveMatomoData:
       call: http.get
       args:
           url: https://.....
       result: matomoData
 
   - insertAll:
           call: googleapis.bigquery.v2.tabledata.insertAll
           args:
               datasetId: myDatasetId
               projectId: myProjectId
               tableId: myTableId
               body:
               "rows": [
           {
        json: should be the full "matomoData" response
       }
]

The response structure of the API I use is:

{
 "body": [
    {
    …
    },
    {
    …
    }
]

(which is an array that corresponds to several rows to insert) It does not match with the structure to insert rows in Bigquery:

"rows": [
           {
        json: …
       },
    json: …
]

Do you have any idea of how can I handle this?



Solution 1:[1]

While the Workflows syntax and standard library can perform simple data extraction and transformation, larger JSON transformations are likely unwieldy inside Workflows for now. I'd recommend using a Cloud Function with a JSON transformation library.

Sources

This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.

Source: Stack Overflow

Solution Source
Solution 1 Kris Braun