'Hadoop 2.7.3 in 【Pseudo-Distributed Operation】When submitting yarn jar,it will tell you ExitCode = 0,container Error。

First,I do it following the document - Pseudo-Distributed Operation

https://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-common/SingleCluster.html

1、My JPS like following

neko@solarisneko:~/Server/BigData/hadoop-2.7.3$ jps
7652 SecondaryNameNode
8184 NodeManager
7386 DataNode
7852 ResourceManager
3757 
8461 Jps

2、core-site.xml

<?xml version="1.0" encoding="UTF-8"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<configuration>
    <property>
        <name>fs.defaultFS</name>
        <value>hdfs://localhost:9000</value>
    </property>
</configuration>
```xml

3、hdfs-site.xml
```xml
<?xml version="1.0" encoding="UTF-8"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<configuration>
    <property>
        <name>dfs.replication</name>
        <value>1</value>
    </property>
</configuration>

4、mapred-site.xml

<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>

<configuration>
    <property>
        <name>mapreduce.framework.name</name>
        <value>yarn</value>
    </property>
    <property>
        <name>mapreduce.application.classpath</name>
        <value>$HADOOP_MAPRED_HOME/share/hadoop/mapreduce/*:$HADOOP_MAPRED_HOME/share/hadoop/mapreduce/lib/*</value>
    </property>
</configuration>
```xml

5、yarn-site.xml
```xml
<?xml version="1.0"?>
<configuration>
    <property>
        <name>yarn.nodemanager.aux-services</name>
        <value>mapreduce_shuffle</value>
    </property>
    <property>
        <name>yarn.nodemanager.env-whitelist</name>
        <value>JAVA_HOME,HADOOP_COMMON_HOME,HADOOP_HDFS_HOME,HADOOP_CONF_DIR,CLASSPATH_PREPEND_DISTCACHE,HADOOP_YARN_HOME,HADOOP_HOME,PATH,LANG,TZ,HADOOP_MAPRED_HOME</value>
    </property>
</configuration>

6、slaves

localhost

7、/etc/hosts

127.0.0.1       localhost
127.0.0.1       neko
127.0.0.1       solarisneko

And then I submit yarn jar

yarn jar share/hadoop/mapreduce/hadoop-mapreduce-examples-2.7.3.jar grep input output 'dfs[a-z.]+'

And it print log

22/02/06 00:37:18 INFO client.RMProxy: Connecting to ResourceManager at /0.0.0.0:8032
22/02/06 00:37:18 INFO input.FileInputFormat: Total input paths to process : 9
22/02/06 00:37:19 INFO mapreduce.JobSubmitter: number of splits:9
22/02/06 00:37:19 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1644079006534_0001
22/02/06 00:37:19 INFO impl.YarnClientImpl: Submitted application application_1644079006534_0001
22/02/06 00:37:19 INFO mapreduce.Job: The url to track the job: http://localhost:8088/proxy/application_1644079006534_0001/
22/02/06 00:37:19 INFO mapreduce.Job: Running job: job_1644079006534_0001
22/02/06 00:37:26 INFO mapreduce.Job: Job job_1644079006534_0001 running in uber mode : false
22/02/06 00:37:26 INFO mapreduce.Job:  map 0% reduce 0%
22/02/06 00:37:33 INFO mapreduce.Job:  map 11% reduce 0%
22/02/06 00:37:34 INFO mapreduce.Job:  map 67% reduce 0%

And then it stuck.

I try to watch nodemanager-xxx.log,then I get this.

2022-02-06 00:37:32,262 WARN org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch: Exception when trying to cleanup container container_1644079006534_0001_01_000006: ExitCodeException exitCode=143: 
    at org.apache.hadoop.util.Shell.runCommand(Shell.java:582)
    at org.apache.hadoop.util.Shell.run(Shell.java:479)
    at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:773)
    at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.killContainer(DefaultContainerExecutor.java:452)
    at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.signalContainer(DefaultContainerExecutor.java:408)
    at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.cleanupContainer(ContainerLaunch.java:419)
    at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainersLauncher.handle(ContainersLauncher.java:139)
    at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainersLauncher.handle(ContainersLauncher.java:55)
    at org.apache.hadoop.yarn.event.AsyncDispatcher.dispatch(AsyncDispatcher.java:184)
    at org.apache.hadoop.yarn.event.AsyncDispatcher$1.run(AsyncDispatcher.java:110)
    at java.base/java.lang.Thread.run(Thread.java:834)

If I update the memory config params, then it will ExitCode = 0, and tell you container is shutdown..



Sources

This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.

Source: Stack Overflow

Solution Source