'How to order sinks correctly in azure dataflow?
I've a complex azure dataflow with about 140 transformations and 21 sinks (inline and cache) and I'm currently getting a warning about custom sink ordering
Some sinks are not ordered correctly in their group. Ensure that all sinks in a group are ordered inbetween sinks outside that group
I've tried to find any info about custom sink ordering or grouping but the documentation is quite sparse.
https://docs.microsoft.com/en-us/azure/data-factory/data-flow-sink#custom-sink-ordering
So I would like to know why my sinks are grouped and what the warning is trying to tell me? The sinks within the group are ordered inbetween the outer sinks (see image).
Solution 1:[1]
As per the shared image, the write order for the sink groups is not in increasing order (6, 7, 8 missing). Make sure the groups and sinks inside the group should be in increasing order.
Refer below image as example.
Check the Custom sink order box and give the order to your sinks so that the data flow follow the exact sink order.
Sources
This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.
Source: Stack Overflow
| Solution | Source |
|---|---|
| Solution 1 | UtkarshPal-MT |