'Databrick: Send specific error message from sub notebook to main notebook that executes the sub notebook with ThreadPoolExecutor()

I am trying to send an error message from a sub notebook to a main notebook that executes the sub notebook with ThreadPoolExecutor().

If I send the message via dbutils.notebook.exit() the result is None, and if I try to raise a specific exception, I get a different error message than the one send from the sub notebook. Is it possible to send a specific error message from the sub notebook to futures from ThreadPoolExecutor()?

Thanks!

#Part of code to execute sub notebook
def submitNotebook_ori(parameters):
    dbutils.notebook.run('sub_bronze_tanaros_error_testing', 1200, parameters)

from concurrent import futures
with futures.ThreadPoolExecutor() as executor:
      results = futures.wait([executor.submit(submitNotebook_ori, parameter) for parameter in parameters])
for result in results.done:
    if result.exception() is not None:
        raise Exception("Either the file is unavailable or bad records exists")
    else:
        print(result.result())

#Error message when recieved
An error occurred while calling o1916._run.
: com.databricks.WorkflowException: com.databricks.NotebookExecutionException: FAILED
    at com.databricks.workflow.WorkflowDriver.run(WorkflowDriver.scala:96)
    at com.databricks.dbutils_v1.impl.NotebookUtilsImpl.run(NotebookUtilsImpl.scala:117)
    at com.databricks.dbutils_v1.impl.NotebookUtilsImpl._run(NotebookUtilsImpl.scala:84)
    at sun.reflect.GeneratedMethodAccessor331.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:244)
    at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:380)
    at py4j.Gateway.invoke(Gateway.java:295)
    at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132)
    at py4j.commands.CallCommand.execute(CallCommand.java:79)
    at py4j.GatewayConnection.run(GatewayConnection.java:251)
    at java.lang.Thread.run(Thread.java:748)
Caused by: com.databricks.NotebookExecutionException: FAILED
    at com.databricks.workflow.WorkflowDriver.run0(WorkflowDriver.scala:142)
    at com.databricks.workflow.WorkflowDriver.run(WorkflowDriver.scala:91)
    ... 12 more'''



#simplified code from sub notebook
try:
    #Read and manage data
except Exception as error:
    #dbutils.notebook.exit("Unable to process file")
    raise Exception("Unable to process file") 


Sources

This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.

Source: Stack Overflow

Solution Source