'Hadoop 2.7.3 in 【Pseudo-Distributed Operation】When submitting yarn jar,it will tell you ExitCode = 0,container Error。
First,I do it following the document - Pseudo-Distributed Operation
https://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-common/SingleCluster.html
1、My JPS like following
neko@solarisneko:~/Server/BigData/hadoop-2.7.3$ jps
7652 SecondaryNameNode
8184 NodeManager
7386 DataNode
7852 ResourceManager
3757
8461 Jps
2、core-site.xml
<?xml version="1.0" encoding="UTF-8"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<configuration>
<property>
<name>fs.defaultFS</name>
<value>hdfs://localhost:9000</value>
</property>
</configuration>
```xml
3、hdfs-site.xml
```xml
<?xml version="1.0" encoding="UTF-8"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<configuration>
<property>
<name>dfs.replication</name>
<value>1</value>
</property>
</configuration>
4、mapred-site.xml
<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<configuration>
<property>
<name>mapreduce.framework.name</name>
<value>yarn</value>
</property>
<property>
<name>mapreduce.application.classpath</name>
<value>$HADOOP_MAPRED_HOME/share/hadoop/mapreduce/*:$HADOOP_MAPRED_HOME/share/hadoop/mapreduce/lib/*</value>
</property>
</configuration>
```xml
5、yarn-site.xml
```xml
<?xml version="1.0"?>
<configuration>
<property>
<name>yarn.nodemanager.aux-services</name>
<value>mapreduce_shuffle</value>
</property>
<property>
<name>yarn.nodemanager.env-whitelist</name>
<value>JAVA_HOME,HADOOP_COMMON_HOME,HADOOP_HDFS_HOME,HADOOP_CONF_DIR,CLASSPATH_PREPEND_DISTCACHE,HADOOP_YARN_HOME,HADOOP_HOME,PATH,LANG,TZ,HADOOP_MAPRED_HOME</value>
</property>
</configuration>
6、slaves
localhost
7、/etc/hosts
127.0.0.1 localhost
127.0.0.1 neko
127.0.0.1 solarisneko
And then I submit yarn jar
yarn jar share/hadoop/mapreduce/hadoop-mapreduce-examples-2.7.3.jar grep input output 'dfs[a-z.]+'
And it print log
22/02/06 00:37:18 INFO client.RMProxy: Connecting to ResourceManager at /0.0.0.0:8032
22/02/06 00:37:18 INFO input.FileInputFormat: Total input paths to process : 9
22/02/06 00:37:19 INFO mapreduce.JobSubmitter: number of splits:9
22/02/06 00:37:19 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1644079006534_0001
22/02/06 00:37:19 INFO impl.YarnClientImpl: Submitted application application_1644079006534_0001
22/02/06 00:37:19 INFO mapreduce.Job: The url to track the job: http://localhost:8088/proxy/application_1644079006534_0001/
22/02/06 00:37:19 INFO mapreduce.Job: Running job: job_1644079006534_0001
22/02/06 00:37:26 INFO mapreduce.Job: Job job_1644079006534_0001 running in uber mode : false
22/02/06 00:37:26 INFO mapreduce.Job: map 0% reduce 0%
22/02/06 00:37:33 INFO mapreduce.Job: map 11% reduce 0%
22/02/06 00:37:34 INFO mapreduce.Job: map 67% reduce 0%
And then it stuck.
I try to watch nodemanager-xxx.log,then I get this.
2022-02-06 00:37:32,262 WARN org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch: Exception when trying to cleanup container container_1644079006534_0001_01_000006: ExitCodeException exitCode=143:
at org.apache.hadoop.util.Shell.runCommand(Shell.java:582)
at org.apache.hadoop.util.Shell.run(Shell.java:479)
at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:773)
at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.killContainer(DefaultContainerExecutor.java:452)
at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.signalContainer(DefaultContainerExecutor.java:408)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.cleanupContainer(ContainerLaunch.java:419)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainersLauncher.handle(ContainersLauncher.java:139)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainersLauncher.handle(ContainersLauncher.java:55)
at org.apache.hadoop.yarn.event.AsyncDispatcher.dispatch(AsyncDispatcher.java:184)
at org.apache.hadoop.yarn.event.AsyncDispatcher$1.run(AsyncDispatcher.java:110)
at java.base/java.lang.Thread.run(Thread.java:834)
If I update the memory config params, then it will ExitCode = 0, and tell you container is shutdown..
Sources
This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.
Source: Stack Overflow
| Solution | Source |
|---|
