Edit Mapred site XML file

If you are using MapReduce, you must edit the mapred-site.xml file to indicate where the job history logs are stored and to allow MapReduce jobs to run across platforms.

Perform the following steps to edit the mapred-site.xml file:

  1. Navigate to the *<username>*/.pentaho/metastore/pentaho/NamedCluster/Configs/*<user-defined connection name>* directory and open the mapred-site.xml file.

  2. Add the following values:

Parameter

Value

mapreduce.jobhistory.address

Set this to the folder where you want to store the job history logs.

mapreduce.application.classpath

Add classpath information. Here is an example:```xmlmapreduce.application.classpath$PWD/mr-framework/hadoop/share/hadoop/mapreduce/* :$PWD/mr-framework/hadoop/share/hadoop/mapreduce/lib/* :$PWD/mr-framework/hadoop/share/hadoop/common/:$PWD/mr-framework/hadoop/share/hadoop/common/lib/ :$PWD/mr-framework/hadoop/share/hadoop/yarn/:$PWD/mr-framework/hadoop/share/hadoop/yarn/lib/ :$PWD/mr-framework/hadoop/share/hadoop/hdfs/:$PWD/mr-framework/hadoop/share/hadoop/hdfs/lib/ :/usr/hdp/${hdp.version}/hadoop/lib/hadoop-lzo-0.6.0.${hdp.version}.jar:/etc/hadoop/conf/secure


</td></tr><tr><td>

**mapreduce.application.framework.path**

</td><td>

Set the framework path. Here is an example:```xml
<property>
  <name>mapreduce.application.framework.path</name>
  <value>/hdp/apps/${hdp.version}/mapreduce/mapreduce.tar.gz#mr-framework</value>
</property>

3. Verify the **mapreduce.app-submission.cross-platform** property is in the `mapred-site.xml` file. If it is not in the file, add it as follows.

Parameter

Value

mapreduce.app-submission.cross-platform

Add this property to allow MapReduce jobs to run on either Windows client or Linux server platforms.```xmlmapreduce.app-submission.cross-platformtrue


</td></tr></tbody>
</table>4.  Save and close the file.

Last updated