'dbt pipeline call from Azure Data Factory using Azure Container Instance with a variable command

I'm looking to call dbt in a child pipeline from Azure Data Factory. Looking at the following project on GitHub it's been done before. https://gitlab.com/hashmapinc/oss/dbtonazure

What I would like to do though is pass a parametrised call from the Data Factory Web Activity so instead of having an Environment Variable in an Azure Containerised Instance and having to have a separate ACI for each dbt command I'd like to run I can pass a variable command to dbt.

e.g. Main pipeline runs and passes a variable "a". dbt runs dbt run --select a

Main pipeline runs and passes a variable "b". dbt runs dbt run --select b

I'm leaning to writing an Azure Function to accept incoming messages, where the message is the command.

Thanks, Dan



Sources

This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.

Source: Stack Overflow

Solution Source