新部署 Hadoop 3.x 提交 MapReduce 时会出现以下报错:
Error: Could not find or load main class org.apache.hadoop.mapreduce.v2.app.MRAppMasterPlease check whether your etc/hadoop/mapred-site.xml contains the below configuration:
<property><name>yarn.app.mapreduce.am.env</name><value>HADOOP_MAPRED_HOME=${full path of your hadoop distribution directory}</value>
</property>
<property><name>mapreduce.map.env</name><value>HADOOP_MAPRED_HOME=${full path of your hadoop distribution directory}</value>
</property>
<property><name>mapreduce.reduce.env</name><value>HADOOP_MAPRED_HOME=${full path of your hadoop distribution directory}</value>
</property>
而网上很多的解决方案是:在mapred-site.xml配置中加入配置信息之后,分发到各个节点,再重启,但依然会有以上问题。
<property><name>yarn.app.mapreduce.am.env</name><value>HADOOP_MAPRED_HOME=${HADOOP_HOME}</value></property><property><name>mapreduce.map.env</name><value>HADOOP_MAPRED_HOME=${HADOOP_HOME}</value></property><property><name>mapreduce.reduce.env</name><value>HADOOP_MAPRED_HOME=${HADOOP_HOME}</value></property>
而报错中最核心的是 Could not find or load main class org.apache.hadoop.mapreduce.v2.app.MRAppMaster
在命令行执行 hadoop classpath
获取 Hadoop classpath
然后在 yarn-site.xml,添加信息:
<property><name>yarn.application.classpath</name><value>复制的Hadoop classpath信息</value></property>