Troubleshooting AEL

Follow the suggestions in these topics to help resolve common issues with running transformations with the Adaptive Execution Layer.

• Steps cannot run in parallel

• Table Input step fails

• User ID below minimum allowed

• Hadoop version conflict

• Hadoop libraries are missing

• Spark libraries conflict with Hadoop libraries

• Failed to find AVRO files

• Unable to access Google Cloud Storage resources

• Unable to access AWS S3 resources

• Internet Address data type fails

• Message size exceeded

• Spark SQL catalyst errors using the Merge or Group By steps

• Performance or memory issues

• Multiple steps in a transformation cannot generate files to the same location

• Cannot read footer in a Spark file

• Errors when using Hive and AEL on a Hortonworks cluster

• Errors when using Hive and AEL on an Amazon EMR cluster

• Driver timeout and deployment errors with AEL on secured clusters

• Steps cannot run with Spark on AEL