# Cautions for Spark tuning

When approaching Spark tuning in PDI, you should consider that one tuning solution may not work for all your PDI transformations. Effective tuning depends on knowing how your environment changes, how your data sizes change, which PDI steps require more resources, and how tuning those steps impacts the execution of the transformation. As a best practice, you should know how your data **skews**. Balanced partitions increase stability, but skewing data too much may cause executors to overflow and fail.

The Spark engine in PDI runs on the Adaptive Execution Layer (AEL), which does not safeguard against incorrect tuning. Some tuning may cause executors and applications to fail, and may reduce performance instead of improving it. See [Optimizing Spark tuning](/install/9.3-install/pentaho-configuration/tasks-to-be-performed-by-an-it-administrator/set-up-the-adaptive-execution-layer-ael/advanced-topics/spark-tuning-landing-page-cp/configuring-application-tuning-parameters-for-spark/optimizing-spark-tuning.md) to learn how to use PDI and Spark logging to monitor changes to application and step tuning parameters, and how those changes affect the execution of your transformation and its related Spark memory model.


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://docs.pentaho.com/install/9.3-install/pentaho-configuration/tasks-to-be-performed-by-an-it-administrator/set-up-the-adaptive-execution-layer-ael/advanced-topics/spark-tuning-landing-page-cp/about-spark-tuning-in-pdi-cp/cautions-for-spark-tuning-about-spark-tuning.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.