Edit Mapred site XML file

If you are using MapReduce, edit the mapred-site.xml file to indicate where the job history logs are stored and to allow MapReduce jobs to run across platforms.

Perform the following steps to edit the mapred-site.xml file:

  1. Navigate to the *<username>*/.pentaho/metastore/pentaho/NamedCluster/Configs/*<user-defined connection name>* directory and open the mapred-site.xml file.

  2. Verify that the mapreduce.jobhistory.address and mapreduce.job.hdfs-servers properties are in the mapred-site.xml file. If they are not in the file, you can add them as follows:

Parameter

Value

mapreduce.jobhistory.address

Set this property to the place where job history logs are stored, as shown in the following example:```xmlmapreduce.jobhistory.address<active node name in the cluster>:10020


</td></tr><tr><td>

**mapreduce.job.hdfs-servers**

</td><td>

Add this property for YARN:

```xml
<property>
   <name>mapreduce.job.hdfs-servers</name>
   <value>hdfs://*&lt;active node name in the cluster&gt;*:8020</value>
</property>

3. Optionally, to allow YARN containers to launch on JDK11.x nodes, add the **mapreduce.jvm.add-opens-as-default** property to the `mapred-site.xml` file, as shown below. All MapReduce jobs require this property to be added in `mapred-site.xml` to run successfully on JDK11.x machines as JAVA 11 does not require **ADD\_OPENS** JVM arguments by default.

**Note:** Do not add this property to containers using JDK17.x nodes.

Parameter

Value

mapreduce.jvm.add-opens-as-default

Add this property for YARN to launch on JDK11.x nodes:```xmlmapreduce.jvm.add-opens-as-default false


</td></tr></tbody>
</table>4.  Save and close the file.

Last updated

Was this helpful?