'Is there a way to check if the Databricks Autoloader wrote to the bad records path?

I am trying to have some form of flag raised whenever a bad record is failed to be read by the autoloader and taken to the bad records path without having to look into the bad records folder. I have read this entire page but there was nothing on the topic of bad records beyond how to set it up.

https://docs.microsoft.com/en-us/azure/databricks/spark/latest/structured-streaming/auto-loader

azurepysparkdatabricksrecordautoload


Sources

This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.

Source: Stack Overflow

Solution Source

Related Questions