Opening the PDI step Spark tuning options

You can access the Spark tuning options from a PDI step by right-clicking on the step in the canvas and selecting Spark tuning parameters from the menu.

Spark Tuning Parameters menu

Note: Spark tuning options are not available for some PDI steps, such as most Consumer steps and the Avro, Hbase, Orc, and Parquet input steps.

The Spark tuning parameters dialog box appears. Click in the column to view the Spark tuning options available for that step in the dialog box. These options vary from step to step, depending on the scope of the step.

Spark tuning dialog box

Note: Support for the step-level Spark tuning options are based on the Spark version installed on the cluster. While Pentaho 9.3 is coded for Spark 2.3, the actual implementation varies with the Spark version running on the cluster.