'spark-submit suddenly shows log4j:WARN
When I try to submit a script via spark-sumbit:
$ spark-submit --driver-memory 110g --master local[*] ./Analysis/myscript.py
from one minute to another this errors are thrown:
log4j:WARN No appenders could be found for logger (org.apache.spark.util.ShutdownHookManager). log4j:WARN Please initialize the log4j system properly. log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.
I did not change anything in the spark configuration. Running the same code via pyspark shell works fine.
Sources
This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.
Source: Stack Overflow
| Solution | Source |
|---|