Http requests in azure data factory to retrieve xml data and store it in azure storage blob

Question

Previously we made logic app in azure where we used http request to retrieve xml file from our clients system. It goes like that: HTTP request --> response body is xml data --> we save that xml data in azure blob storage as xml file. My question is how and if its possible to do the same thing in azure data factory? Reason for us to move this process over to data factory is that we also need to execute sql server stored procedures there and in logic app there is that 2 minute timeout and some of our procedures run longer than 2 min.

Answer 1

If you're looking for a way to manually trigger an Azure Data Factory pipeline,

You can manually run your pipeline by using one of the following methods:

• .NET SDK
• Azure PowerShell module
• REST API
• Python SDK

The following sample command shows you how to run your pipeline by using the REST API manually:

POST
https://management.azure.com/subscriptions/mySubId/resourceGroups/myResourceGroup/providers/Microsoft.DataFactory/factories/myDataFactory/pipelines/copyPipeline/createRun?api-version=2017-03-01-preview

More information: Manual execution (on-demand) with JSON

There are more questions to be answered, however, like "can we increase the timeout for the Logic App" (yes, see HTTP request limits - Timeout duration), "does the Logic App need to wait for the Stored Procedures to complete" and "is Data Factory the best tool for the job". The best answer to your question depends on the answer to all of these questions.

Based on the information you provided, running the logic in a different way like a Logic App on an Integrated Service Environment or an Azure Function feels like the best option.