Edit Mapred site XML file
If you are using MapReduce, edit the mapred-site.xml
file to indicate where the job history logs are stored and to allow MapReduce jobs to run across platforms.
Perform the following steps to edit the mapred-site.xml
file:
Navigate to the
*<username>*/.pentaho/metastore/pentaho/NamedCluster/Configs/*<user-defined connection name>*
directory and open themapred-site.xml
file.Verify that the mapreduce.jobhistory.address and mapreduce.job.hdfs-servers properties are in the
mapred-site.xml
file. If they are not in the file, you can add them as follows:
Parameter
Value
mapreduce.jobhistory.address
Set this property to the place where job history logs are stored, as shown in the following example:```xmlmapreduce.jobhistory.address<active node name in the cluster>:10020
</td></tr><tr><td>
**mapreduce.job.hdfs-servers**
</td><td>
Add this property for YARN:
```xml
<property>
<name>mapreduce.job.hdfs-servers</name>
<value>hdfs://*<active node name in the cluster>*:8020</value>
</property>
3. Optionally, to allow YARN containers to launch on JDK11.x nodes, add the **mapreduce.jvm.add-opens-as-default** property to the `mapred-site.xml` file, as shown below. All MapReduce jobs require this property to be added in `mapred-site.xml` to run successfully on JDK11.x machines as JAVA 11 does not require **ADD\_OPENS** JVM arguments by default.
**Note:** Do not add this property to containers using JDK17.x nodes.
Parameter
Value
mapreduce.jvm.add-opens-as-default
Add this property for YARN to launch on JDK11.x nodes:```xmlmapreduce.jvm.add-opens-as-default false
</td></tr></tbody>
</table>4. Save and close the file.
Last updated
Was this helpful?