'DIU does not increase not more than 4 on copy activity
I am trying to copy data from GCP(Big Query) Azure Storage Gen2 parquet file with below configuration. Increased DIU from 4 to 16 but during runtime the DIU does not go beyond 4. Can you please help on how to increase the DIU to make my process faster?
- using preserve hierarchy
- Data size 12 millions with 3gb
- Throughput is 2.5mbps
Solution 1:[1]
To increase DIU for a copy activity just click on the activity, and under the Settings tab you can find the Data Integration Unit selector.
Solution 2:[2]
I have the same issue. My source is Azure Blob (csv files), Staging is also Azure Blob and final destination is Snowflake with Azure. All in the same region/zone.
I have set DIU for 20 and Parallel to 4. But only has DIU as 4 utilized and Parallel to 1.
Sources
This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.
Source: Stack Overflow
| Solution | Source |
|---|---|
| Solution 1 | Martin Esteban Zurita |
| Solution 2 | Asad Khan |