'Why does my MapReduce job get stuck at Map 0% Reduce 0%?
I'm testing a filesystem on Hadoop 3.3.1, and every time I run a MapReduce job it gets stuck at "map 0% reduce 0%", meanwhile everything (containers, applications, etc.) is stuck at "RUNNING". Does anybody know why? Or what should I do to find out what the real problem is?
Here's the console output:
hadoop jar share/hadoop/mapreduce/hadoop-mapreduce-examples-3.3.1.jar wordcount wordcount/input wordcount/output
2022-02-27 15:17:21,136 INFO client.DefaultNoHARMFailoverProxyProvider: Connecting to ResourceManager at master/192.168.1.1:8032
2022-02-27 15:17:21,731 INFO input.FileInputFormat: Total input files to process : 5
2022-02-27 15:17:21,761 INFO mapreduce.JobSubmitter: number of splits:5
2022-02-27 15:17:22,064 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1645945958451_0002
2022-02-27 15:17:22,064 INFO mapreduce.JobSubmitter: Executing with tokens: []
2022-02-27 15:17:22,241 INFO conf.Configuration: resource-types.xml not found
2022-02-27 15:17:22,242 INFO resource.ResourceUtils: Unable to find 'resource-types.xml'.
2022-02-27 15:17:22,498 INFO impl.YarnClientImpl: Submitted application application_1645945958451_0002
2022-02-27 15:17:22,539 INFO mapreduce.Job: The url to track the job: http://master:8088/proxy/application_1645945958451_0002/
2022-02-27 15:17:22,540 INFO mapreduce.Job: Running job: job_1645945958451_0002
2022-02-27 15:17:27,644 INFO mapreduce.Job: Job job_1645945958451_0002 running in uber mode : false
2022-02-27 15:17:27,646 INFO mapreduce.Job: map 0% reduce 0%
Here is my yarn-site.xml configuration:
<configuration>
<property>
<name>yarn.resourcemanager.hostname</name>
<value>master</value>
</property>
<property>
<name>yarn.nodemanager.aux-services</name>
<value>mapreduce_shuffle</value>
</property>
<property>
<name>yarn.nodemanager.resource.cpu-vcores</name>
<value>8</value>
</property>
<property>
<name>yarn.nodemanager.disk-health-checker.max-disk-utilization-per-disk-percentage</name>
<value>99.0</value>
</property>
<property>
<name>fs.AbstractFileSystem.madfs.impl</name>
<value>madfs.AbsXxxFS</value>
</property>
</configuration>
My mapred-site.xml configuration:
<configuration>
<property>
<name>mapreduce.framework.name</name>
<value>yarn</value>
</property>
<property>
<name>mapreduce.jobhistory.address</name>
<value>master:10020</value>
</property>
<property>
<name>yarn.app.mapreduce.am.env</name>
<value>HADOOP_MAPRED_HOME=/home/xyz/hadoop-3.3.1</value>
</property>
<property>
<name>mapreduce.map.env</name>
<value>HADOOP_MAPRED_HOME=/home/xyz/hadoop-3.3.1</value>
</property>
<property>
<name>mapreduce.reduce.env</name>
<value>HADOOP_MAPRED_HOME=/home/xyz/hadoop-3.3.1</value>
</property>
</configuration>
P.s. there's no ERROR nor WARNING in the logs dumped by ResourceManager and NodeManager. And the states of RMAppAttemptImpl, RMAppImpl, RMContainerImpl, ContainerMonitorImpl and ApplicationImpl are all stuck at RUNNING.
Here's the ResourceManager log hadoop-resourcemanager.out, and here's the NodeManager log hadoop-nodemanager.out.
Update:
The problem seems to be the application failed to preempt the required resources...
The app status I got using yarn app -status <appId>:
Application Report :
Application-Id : application_1645945958451_0003
Application-Name : QuasiMonteCarlo
Application-Type : MAPREDUCE
User : xyz
Queue : default
Application Priority : 0
Start-Time : 1645976932440
Finish-Time : 0
Progress : 0%
State : RUNNING
Final-State : UNDEFINED
Tracking-URL : http://master:45299
RPC Port : 46709
AM Host : master
Aggregate Resource Allocation : 997310 MB-seconds, 486 vcore-seconds
Aggregate Resource Preempted : 0 MB-seconds, 0 vcore-seconds
Log Aggregation Status : DISABLED
Diagnostics :
Unmanaged Application : false
Application Node Label Expression : <Not set>
AM container Node Label Expression : <DEFAULT_PARTITION>
TimeoutType : LIFETIME ExpiryTime : UNLIMITED RemainingTime : -1seconds
Also, only 1 container is allocated to this job, does that has anything to do with the problem?
Sources
This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.
Source: Stack Overflow
| Solution | Source |
|---|
