'Data Factory - Can I use the date field in a CSV to determine the destination folder in Copy Activity

I have some CSV files that I want to copy to a specific folder in ADLS based on the date column within the file.

i.e. CSV file has a column named "date" that reads "2022-02-23" on all rows. I want to copy that file to a folder that has the corresponding year and month, such as "/curated/UK/ProjectABC/2022/02"

I've got a Lookup activity that's pointing to the source CSV file and populating a Set Variable activity with the month using this dynamic content - @substring(string(activity('Lookup1').output.firstrow.date),5,2)

Would this be the right approach, to use a variable?

I cant use variables in the Directory portion of the Sink Dataset, as far as I know.

Have you come across this situation before?



Sources

This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.

Source: Stack Overflow

Solution Source