Azure Stream Analytic Output

Question

Azure Stream Analytic storing directory like this. Earlier we used datepath (2022/04/14) for path pattern and its output store separate directory in datalake. Now it has changed storing directory like '2022%2F4%2F19'. How to solve this problem?

Answer 1

When you configure the Azure Stream Analytic Output you can mention the Path pattern, Date format, and Time format. All are optional properties we can use to filter out.

Path pattern

• You can mention the Path pattern which is used to locate the blob within the specified container.
• Do not include a path pattern if you wish to read blobs from the container's root.
• You can specify one or more instances of the following three variables within the path:{date}, {time}, or {partition}`

Example : cluster1/logs/{date}/{time}/{partition}

Date format

The date format in which the files are structured if you utilize the date variable in the path. Example: YYYY/MM/DD

Time format

The date format in which the files are structured if you utilize the Time variable in the path. Currently, it supports only HH (Hours).

References:

• Configure the blob storage as a stream
• SO Thread for Dynamic Path pattern.
• Stream analysis Custom path pattern