'hyperopt - ISSUE of Using SparkTrials on hyperopt when tuning hyperparameter in parallel on Spark environment (Databricks)
When tuning hyperparameters, I tried to use sparkTrials function of hyperopt to tune them in parallel faster, but errors continue to occur.
The error is as follows and is presumed to be related to the spark environment. I can't judge exactly what the error is.
error is ...
Broadcast variable {} not loaded !
I am referring to the Notebook provided by databricks where 'Step 4: Train the Model', and even though it is the same code, errors continue to occur in my databricks environment.
How should I edit my code?
Sources
This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.
Source: Stack Overflow
| Solution | Source |
|---|