> For the complete documentation index, see [llms.txt](https://docs.pentaho.com/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.pentaho.com/install/legacy-redirects/hyperscalers-legacy/installing-pentaho-on-aws.md).

# Installing Pentaho on AWS

This content has moved to [Hyperscalers](broken://spaces/qfaQ2p0JAZrP8b3cpM9a/pages/hkuqBdPGV03lgXHeNZ2R).

Deploy Pentaho on Amazon Web Services (AWS).

### Choose a deployment option

* [Install the Platform Server or PDI Server on AWS](#install-the-platform-server-or-pdi-server-on-aws)
* [Install the Carte Server on AWS](#install-the-carte-server-on-aws)
* [Run PDI-CLI on AWS](#run-pdi-cli-on-aws)
* [Worksheet for AWS hyperscaler](#worksheet-for-aws-hyperscaler)

### Common prerequisites

* An AWS account
* Docker installed on your workstation
* AWS CLI installed on your workstation

Each deployment option can have extra prerequisites.

### Install the Platform Server or PDI Server on AWS

Use these steps to deploy Docker images of the Pentaho Platform Server or PDI Server on AWS.

This workflow uses Amazon EKS, Amazon ECR, and (optionally) Amazon S3.

#### Before you begin

**Prerequisites**

Meet these prerequisites before you start:

* Install a stable version of Docker on your workstation.
* Have an AWS account.
* Install the Amazon AWS CLI on your workstation.
* Review the supported versions:

  | Application | Supported version                   |
  | ----------- | ----------------------------------- |
  | EKS         | v1.x                                |
  | Docker      | v20.10.21 or a later stable version |
  | AWS CLI     | v2.x                                |
  | Python      | v3.x                                |
* Fill in [Worksheet for AWS hyperscaler](#worksheet-for-aws-hyperscaler). You will reuse these values later.

#### Process overview

Use these steps to deploy the Platform Server or PDI Server on AWS:

1. Download and extract Pentaho for AWS.
2. Create an Amazon ECR repository.
3. Load and push the Pentaho Docker image to ECR.
4. Create an RDS database.
5. (Optional) Create an S3 bucket.
6. Create an EKS cluster and add a node group.
7. Install the Platform or PDI Server.

You can also dynamically update server configuration content from S3.

{% stepper %}
{% step %}

### Step 1: Download and extract Platform or PDI Server for AWS

Download and open the package files that contain the files you need to install Pentaho.

1. Navigate to the [Support Portal](https://support.pentaho.com/hc/en-us) and download the AWS version of the Docker image with the corresponding license file for the applications you want to install on your workstation.
2. Extract the image to view the directories and the README file.

   The image package file (`<package-name>.tar.gz`) contains the following:

   | Directory or file name | Content description                                                                     |
   | ---------------------- | --------------------------------------------------------------------------------------- |
   | `image`                | Directory containing all the Pentaho source images.                                     |
   | `sql-scripts`          | Directory containing SQL scripts for various operations.                                |
   | `yaml`                 | Directory containing YAML configuration files and various utility files.                |
   | `README.md`            | File containing a link to detailed information about what is provided for this release. |

{% endstep %}

{% step %}

### Step 2: Create an Amazon ECR

Before pushing the Pentaho image to AWS, create an Amazon ECR repository.

1. Create an ECR repository to load the Pentaho image.

   For details, see [instructions for creating a private repository](https://docs.aws.amazon.com/AmazonECR/latest/userguide/repository-create.html) in AWS docs.
2. Record the name of the ECR repository in [Worksheet for AWS hyperscaler](#worksheet-for-aws-hyperscaler).
   {% endstep %}

{% step %}

### Step 3: Load and push the Pentaho Docker image to ECR

Select and tag the Pentaho Docker image, then push it to the ECR registry.

1. Navigate to the `image` directory containing the Pentaho `.tar.gz` files.
2. Select and load the `.tar.gz` file into the local registry:

   ```
   docker load -i <pentaho-image>.tar.gz
   ```
3. Record the name of the source image that was loaded into the registry:

   ```
   docker images
   ```
4. Tag the source image so it can be pushed to AWS:

   ```
   docker tag <source-image>:<tag> <target-repository>:<tag>
   ```
5. Push the image file into the ECR registry:

   ```
   docker push <target-repository>:<tag>
   ```

   The AWS Management Console displays the uploaded image URI.

   For general instructions, see [Pushing a Docker image](https://docs.aws.amazon.com/AmazonECR/latest/userguide/docker-push-ecr-image.html).
6. Record the newly created ECR repository URI in [Worksheet for AWS hyperscaler](#worksheet-for-aws-hyperscaler).
   {% endstep %}

{% step %}

### Step 4: Create an RDS database

Use these instructions to create a Relational Database Service (RDS) database in AWS.

1. Create an RDS PostgreSQL database for Pentaho to use.

   See the AWS instructions at [Creating and connecting to a PostgreSQL DB instance](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/CHAP_GettingStarted.CreatingConnecting.PostgreSQL.html) and apply the settings in the table below.

   | Section                 | Actions                                                                                                                                                                                                                                                                                                                                                                                                                                           |
   | ----------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
   | Create database         | <p>Choose <strong>Standard create</strong>.</p><p>Select the <strong>PostgreSQL</strong> engine.</p><p>Set the engine version to a PostgreSQL version supported by the <strong>Components reference</strong> found in the <strong>Try Pentaho Data Integration and Analytics</strong> document, such as <strong>PostgreSQL 13.5-R1</strong>.</p>                                                                                                  |
   | Templates               | <p>It is recommended to select the <strong>Free tier</strong> option.</p><p><strong>Note:</strong> For this installation, the <strong>Free tier</strong> PostgreSQL database is used with a set of options as an example. However, you are free to use other database servers with different options as necessary.</p>                                                                                                                            |
   | Settings                | <p>Set the DB instance identifier.</p><p>Retain the default user name <strong>postgres</strong> and set the <strong>Master password</strong>.</p><p>Use the default password authentication setting.</p><p>Use the default values for the rest of the settings in this section.</p>                                                                                                                                                               |
   | Instance configuration  | Use the default settings for each section.                                                                                                                                                                                                                                                                                                                                                                                                        |
   | Storage                 | Use the default settings for each section.                                                                                                                                                                                                                                                                                                                                                                                                        |
   | Connectivity            | <p>Set the <strong>Virtual private cloud (VPC)</strong> and the <strong>DB subnet group</strong> to any of the options available to you. If in doubt, use the default values.</p><p>Select <strong>Public access</strong>.</p><p>Make sure that the VPC security groups selected have a rule enabling communication to the database through the PostgreSQL port, which is 5432 by default.</p><p>For other options, use the default settings.</p> |
   | Database authentication | Use the default setting **Password authentication**.                                                                                                                                                                                                                                                                                                                                                                                              |
2. Run the scripts in the `sql-scripts` folder in the distribution in numeric order.
3. From the AWS **Management Console > Connection & security** tab, record the database **Endpoint** and **Port** number in [Worksheet for AWS hyperscaler](#worksheet-for-aws-hyperscaler).
   {% endstep %}

{% step %}

### Step 5 (Optional): Create an S3 bucket

Create an S3 bucket only if you want to do one or more of the following actions. Otherwise, go to Step 6.

* Add third-party JAR files like JDBC drivers or custom JAR files.
* Customize the default Pentaho configuration.
* Replace server files.
* Upload or update the metastore.
* Add files to the Platform and PDI Server's `/home/pentaho/.kettle` directory.

  This directory is mapped to the `KETTLE_HOME_DIR` environment variable. The `content-config.properties` file uses it.

1. Create an S3 bucket.

   To create an S3 bucket, see [Creating a bucket](https://docs.aws.amazon.com/AmazonS3/latest/userguide/create-bucket-overview.html).

   To upload a file to S3, see [Uploading objects](https://docs.aws.amazon.com/AmazonS3/latest/userguide/upload-objects.html).
2. Record the newly created S3 bucket name in [Worksheet for AWS hyperscaler](#worksheet-for-aws-hyperscaler).
3. Upload files into the S3 bucket.

   After the S3 bucket is created, manually create any needed directories and upload files by using the AWS Management Console.

   The following table lists the relevant Pentaho directories and actions for each directory.

   | Directory                    | Actions                                                                                                                                                                                                                                                                                                                                                                                                             |
   | ---------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
   | `/root`                      | <p>All files in the S3 bucket are copied to the Platform and PDI Server's <code>/home/pentaho/.kettle</code> directory.</p><p>If you must copy a file to the <code>/home/pentaho/.kettle</code> directory, drop the file in the <code>root</code> directory of the S3 bucket.</p>                                                                                                                                   |
   | `custom-lib`                 | <p>If Pentaho needs custom JAR libraries, add the <code>custom-lib</code> directory to the S3 bucket and place the libraries there.</p><p>Any files within this directory will be copied to Pentaho’s <code>lib</code> directory.</p>                                                                                                                                                                               |
   | `jdbc-drivers`               | <p>If the Pentaho installation needs JDBC drivers, do the following:</p><ol><li>Add the <code>jdbc-drivers</code> directory to the S3 bucket.</li><li>Place the drivers in this directory.<br>Any files within this directory will be copied to Pentaho’s <code>lib</code> directory.</li></ol>                                                                                                                     |
   | `plugins`                    | <p>If the Pentaho installation needs additional plugins installed, do the following:</p><ol><li>Add the <code>plugins</code> directory to the S3 bucket.</li><li>Copy the plugins to the <code>plugins</code> directory.<br>Any files within this directory are copied to Pentaho’s <code>plugins</code> directory. For this reason, organize plugins in their own directories, as Pentaho expects.</li></ol>       |
   | `drivers`                    | <p>If the Pentaho installation needs big data drivers installed, do the following:</p><ol><li>Add the <code>drivers</code> directory to the S3 bucket.</li><li>Place the big data drivers in this directory.<br>Any files placed within this directory will be copied to Pentaho’s <code>drivers</code> directory.</li></ol>                                                                                        |
   | `metastore`                  | <p>Pentaho can execute jobs and transformations. Some require additional information that is usually stored in the Pentaho metastore.</p><p>If you must provide the Pentaho metastore to Pentaho, copy the local <code>metastore</code> directory to the root of the S3 bucket. From there, the <code>metastore</code> directory is copied to the proper location within the Docker image.</p>                      |
   | `server-structured-override` | <p>Use <code>server-structured-override</code> only if other mechanisms do not work.</p><p>For example, you can use it for configuring authentication and authorization.</p><p>Any files and directories within this directory will be copied into the <code>pentaho-server</code> directory the same way they appear here.</p><p>If the same files exist in <code>pentaho-server</code>, they are overwritten.</p> |

   The following table lists relevant Pentaho files and actions for each file.

   | File                        | Actions                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                               |
   | --------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
   | `context.xml`               | <p>The Pentaho configuration YAML is included with the image in the <code>templates</code> project directory and is used to install this product. You must set the RDS host and RDS port parameters when you install Pentaho. During installation, the parameters in the YAML are used to generate a custom <code>context.xml</code> so the server can connect to the database repository.</p><p>If these are the only changes required in <code>context.xml</code>, you don’t need to provide a <code>context.xml</code> in S3.</p><p>If you need additional <code>context.xml</code> changes, provide your own <code>context.xml</code> in S3.</p><p>In the <code>context.xml</code> template, replace the <code>\<RDS\_HOST\_NAME></code> and <code>\<RDS\_PORT></code> entries with the values in <a href="#worksheet-for-aws-hyperscaler">Worksheet for AWS hyperscaler</a>.</p> |
   | `content-config.properties` | <p>The <code>content-config.properties</code> file tells the Pentaho Docker image which S3 files to copy and where to place them.</p><p>Each instruction is a line in this format:</p><p><code>${KETTLE\_HOME\_DIR}/\<some-dir-or-file>=${SERVER\_DIR}/\<some-dir></code></p><p>A template for this file is in the templates project directory.</p><p>The template has an entry where <code>context.xml</code> is copied to the required location:</p><p><code>${KETTLE\_HOME\_DIR}/context.xml=${SERVER\_DIR}/tomcat/webapps/pentaho/META-INF/context.xml</code></p>                                                                                                                                                                                                                                                                                                                 |
   | `content-config.sh`         | <p>A bash script that can configure files, change file and directory ownership, move files, install missing apps, and so on.</p><p>Add the script to the S3 bucket.</p><p>The script runs in the Docker image after the other files are processed.</p>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                |
   | `metastore.zip`             | <p>Pentaho can execute jobs and transformations. Some require additional information that is usually stored in the Pentaho metastore.</p><p>If you must provide the Pentaho metastore to Pentaho, zip the content of the <code>local.pentaho</code> directory with the name <code>metastore.zip</code> and add it to the root of the S3 bucket. The <code>metastore.zip</code> file is extracted to the proper location within the Docker image.</p><p><strong>Note:</strong> You cannot copy VFS connections to the hyperscaler server the same way as named connections. Connect to Pentaho on the hyperscaler and create the VFS connection there.</p>                                                                                                                                                                                                                             |

For instructions on how to dynamically update server configuration content from the S3 bucket, see [Dynamically update server configuration content from S3](#dynamically-update-server-configuration-content-from-s3).
{% endstep %}

{% step %}

### Step 6: Create an EKS cluster and add a node group

Use Amazon Elastic Kubernetes Service (EKS) to create a cluster for running the Platform or PDI Server.

1. Create an EKS cluster on AWS.

   For instructions, see [Create an Amazon EKS cluster](https://docs.aws.amazon.com/eks/latest/userguide/create-cluster.html).

   For an introduction to EKS, see [Getting started with Amazon EKS](https://docs.aws.amazon.com/eks/latest/userguide/getting-started.html).

   For information about creating roles to delegate permissions to an AWS service, see [Create a role](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_create_for-service.html).

   | Settings                                              | Actions                                                                                                                                                                                                                                                                                                                   |
   | ----------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
   | Cluster service role                                  | <p>Select any existing role, as long as these policies are attached:</p><ul><li>AmazonEKSClusterPolicy</li><li>AmazonS3FullAccess</li><li>AmazonEKSServicePolicy</li></ul>                                                                                                                                                |
   | VPC                                                   | <p>In the Networking section, do the following:</p><ol><li>Select an existing VPC. The selected VPC populates a group of subnets. Create it before you create a computing or cloud stack.</li><li>Make sure that <strong>Auto-assign public IPv4 address</strong> under Subnets is set to <strong>Yes</strong>.</li></ol> |
   | Cluster endpoint access                               | Select **Public and private**.                                                                                                                                                                                                                                                                                            |
   | <p>Amazon VPC CNI</p><p>CoreDNS</p><p>kube\_proxy</p> | Select all three EKS add-ons with their default configurations.                                                                                                                                                                                                                                                           |
2. Record the newly created EKS cluster name in [Worksheet for AWS hyperscaler](#worksheet-for-aws-hyperscaler).
3. On the **Compute** tab under Node groups, select **Add node group**.

   <div data-gb-custom-block data-tag="hint" data-style="info" class="hint hint-info"><p>The EKS cluster must be in <strong>Active</strong> state before you create nodes.</p></div>

   For more details, see [Create a managed node group](https://docs.aws.amazon.com/eks/latest/userguide/create-managed-node-group.html).
4. In **Node group configuration**, set the node group **Name**.
5. Select a Node IAM role or create a new one. Make sure the role includes these policies:
   * AmazonS3FullAccess
   * AmazonEC2ContainerRegistryReadOnly
   * AmazonEKSWorkerNodePolicy
   * AmazonEKS\_CNI\_Policy
6. Set the instance type to one that has at least 8 GB of memory.
7. In **Node group scaling configuration**, set **Desired size**, **Minimum size**, and **Maximum size**.
8. In **Node group network configuration**, select the subnets for your node group.
9. For the subnets, set **Auto-assign public IPv4 address** to **Yes**.

   For details, see [IP addressing for your VPCs and subnets](https://docs.aws.amazon.com/vpc/latest/userguide/vpc-ip-addressing.html#subnet-public-ip).
10. Select a load balancer.

    For instructions on how to create an AWS Application Load Balancer, see [Application load balancing on Amazon EKS](https://docs.aws.amazon.com/eks/latest/userguide/alb-ingress.html).
    {% endstep %}

{% step %}

### Step 7: Install the Platform or PDI Server on AWS

When your AWS environment is configured, install the Platform Server or PDI Server.

1. Retrieve the kubeconfig from the EKS cluster.

   In your workstation console, run:

   ```
   aws eks update-kubeconfig --name <my_eks_cluster_name> --region <my_EKS_region>
   ```
2. To configure the Platform or PDI Server YAML file, open `pentaho-server-aws-rds-<lb-type>.yaml` in the `yaml` project directory.

   | lb-type | When to use                                                  |
   | ------- | ------------------------------------------------------------ |
   | alb     | Use this if you installed the AWS Application Load Balancer. |
   | nginx   | Use this if you installed the NGINX Ingress Controller.      |
3. Update the YAML file by copying the values you recorded in [Worksheet for AWS hyperscaler](#worksheet-for-aws-hyperscaler).
4. Retrieve the Platform or PDI Server entry point URI.

   Run either command on your workstation:

   ```
   kubectl get ingress -n pentaho-server
   ```

   or:

   ```
   echo $( kubectl get ingress -n pentaho-server -o jsonpath='{.items..hostname}' )
   ```

   The default port is 80.
5. Deploy the Platform or PDI Server:

   ```
   kubectl apply -f <path to Pentaho deployment YAML>
   ```
6. Test the Platform or PDI Server by retrieving the LoadBalancer Ingress URI:

   ```
   echo $( kubectl get ingress -n pentaho-server -o jsonpath='{.items..hostname}' )
   ```

   <div data-gb-custom-block data-tag="hint" data-style="info" class="hint hint-info"><p>The port number for this load balancer is 80, not 8080.</p></div>
7. Open the URI in a Pentaho-supported browser and sign in.

   | Field    | Default value |
   | -------- | ------------- |
   | Username | admin         |
   | Password | password      |

{% endstep %}
{% endstepper %}

#### Dynamically update server configuration content from S3

If the S3 bucket changed and you need to reflect these changes in the Platform or PDI Server, use these steps.

Before you deploy the Platform or PDI Server, set **allow\_live\_config** to `true` in `pentaho-server-aws-rds.yaml`.

1. Navigate to the directory that contains the configuration you want to update.
2. Prepare the update script by setting `<config_command>` to one of these values:

   | Command option | Description                                                                                  |
   | -------------- | -------------------------------------------------------------------------------------------- |
   | load\_from\_s3 | Copies the content of the bucket to the server’s `/home/pentaho/.kettle` directory.          |
   | restart        | Restarts the Platform or PDI Server without restarting the pod.                              |
   | update\_config | Runs `load_from_s3`, runs all configuration and initialization scripts, then runs `restart`. |

   <div data-gb-custom-block data-tag="hint" data-style="warning" class="hint hint-warning"><p><code>restart</code> and <code>update_config</code> disrupt sticky sessions and impact user sessions.</p></div>
3. Run the configuration update script.

   <div data-gb-custom-block data-tag="hint" data-style="info" class="hint hint-info"><p>If you have multiple Platform or PDI Server replicas, remove the comment (<code>#</code>) in front of <code>sleep 60</code>.</p></div>

   ```
   for pod in $( kubectl get pods -o name -n pentaho-server )
   do
     	echo "Forwarding port on pod: $pod"
    	 pid=$( kubectl port-forward -n pentaho-server $pod 8090:8090 1>/dev/null & echo $! )
    	 while ! nc -z localhost 8090; do 
      		 sleep 0.1
     	done
    	 echo "Executing command ..."
    	 result=$( curl http://localhost:8090/<config_command> )
    	 echo "Command result: $result"
     echo "Killing port forward pid: $pid"
     	while $(kill -9 $pid 2>/dev/null); do 
     	  	sleep 1
    	done
     	# sleep 60
   done;
   ```
4. Verify that the servers restart properly.

### Install the Carte Server on AWS

These instructions help you deploy Docker images of the Carte Server on AWS.

#### Prerequisites

Meet these requirements before you start:

* Install a stable version of Docker on your workstation.
* Have an AWS account.
* Install the AWS CLI on your workstation.

Supported versions:

* Amazon EKS: `v1.x`
* Docker: `v20.10.21` or later stable version
* AWS CLI: `v2.x`

#### Process overview

{% stepper %}
{% step %}

### Step 1: Download and extract Pentaho for AWS

1. Go to the [Support Portal](https://support.pentaho.com/hc/en-us).
2. Download the AWS Docker image package and license file you need.
3. Extract the archive.

The package contains:

* `image/`: Pentaho source images
* `yaml/`: YAML configuration files and utility files
* `README.md`: link to release details
  {% endstep %}

{% step %}

### Step 2: Create an Amazon ECR

Create an ECR repository for the Pentaho image.

1. Follow AWS guidance for [creating a private repository](https://docs.aws.amazon.com/AmazonECR/latest/userguide/repository-create.html).
2. Record the repository name in the [Worksheet for AWS hyperscaler](#worksheet-for-aws-hyperscaler).
   {% endstep %}

{% step %}

### Step 3: Load and push the Pentaho Docker image to ECR

Select and tag the Pentaho Docker image, then push it to ECR.

1. Go to the `image/` directory that contains the Pentaho `tar.gz` files.
2. Load the `tar.gz` file into your local registry:

   ```
   docker load -i <pentaho-image>.tar.gz
   ```
3. List images and note the source image name:

   ```
   docker images
   ```
4. Tag the source image:

   ```
   docker tag <source-image>:<tag> <target-repository>:<tag>
   ```
5. Push the image:

   ```
   docker push <target-repository>:<tag>
   ```

   The AWS Management Console shows the uploaded image URI.
6. Record the ECR repository URI in the [Worksheet for AWS hyperscaler](#worksheet-for-aws-hyperscaler).

For AWS instructions, see [Pushing a Docker image](https://docs.aws.amazon.com/AmazonECR/latest/userguide/docker-push-ecr-image.html).
{% endstep %}

{% step %}

### Step 4: Create an S3 bucket for the Carte Server

Create an S3 bucket for files the container needs at startup.

1. Create an S3 bucket.

   See AWS docs: [Creating a bucket](https://docs.aws.amazon.com/AmazonS3/latest/userguide/create-bucket-overview.html).
2. Record the bucket name in the [Worksheet for AWS hyperscaler](#worksheet-for-aws-hyperscaler).
3. Upload the required directories and files.

   See AWS docs: [Uploading objects](https://docs.aws.amazon.com/AmazonS3/latest/userguide/upload-objects.html).

**S3 bucket directories**

Create these directories in the bucket as needed:

* `root/`
  * Files in this directory are copied to `/home/pentaho/.kettle` in the container.
* `jdbc-drivers/`
  * Put JDBC drivers here.
  * Files are copied to Pentaho’s `lib` directory.
* `plugins/`
  * Put additional plugins here.
  * Files are copied to Pentaho’s `plugins` directory.
  * Organize each plugin in its own directory.

**S3 bucket files**

Upload these files as needed:

* `content-config.properties`
  * Controls which S3 files are copied and where.
  * Add one line per copy instruction:

    ```
    ${KETTLE_HOME_DIR}/<some-dir-or-file>=${APP_DIR}/<some-dir>
    ```
  * Example from the template:

    ```
    ${KETTLE_HOME_DIR}/context.xml=${APP_DIR}/context.xml
    ```
* `content-config.sh`
  * Optional script to configure files, change ownership, install missing apps, and more.
  * Runs after the other files are processed.
    {% endstep %}
    {% endstepper %}

### Run PDI-CLI on AWS

Use the PDI-CLI Docker image to run `kitchen.sh` (transformations) and `pan.sh` (jobs) on AWS.

#### Prerequisites

Meet these requirements before you start:

* Install a stable version of Docker on your workstation.
* Have an AWS account.
* Install the AWS CLI on your workstation.

Supported versions:

* Docker: `v20.10.21` or later stable version
* AWS CLI: `v2.x`

#### Process overview

{% stepper %}
{% step %}

### Step 1: Download and extract Pentaho for AWS

1. Go to the [Support Portal](https://support.pentaho.com/hc/en-us).
2. Download the AWS Docker image package and license file you need.
3. Extract the archive.

The package contains:

* `image/`: Pentaho source images
* `yaml/`: YAML configuration files and utility files
* `README.md`: link to release details
  {% endstep %}

{% step %}

### Step 2: Create an Amazon ECR

Create an ECR repository for the PDI-CLI image.

1. Follow AWS guidance for [creating a private repository](https://docs.aws.amazon.com/AmazonECR/latest/userguide/repository-create.html).
2. Record the repository URI in the [Worksheet for AWS hyperscaler](#worksheet-for-aws-hyperscaler).
   {% endstep %}

{% step %}

### Step 3: Load and push the PDI-CLI Docker image to ECR

Load the image locally, tag it, then push it to ECR.

1. Go to the `image/` directory that contains the PDI-CLI `tar.gz` file.
2. Load the image into your local registry:

   ```
   docker load -i <pdi-cli-image>.tar.gz
   ```
3. List images and note the source image name:

   ```
   docker images
   ```
4. Tag the source image:

   ```
   docker tag <source-image>:<tag> <target-repository>:<tag>
   ```
5. Push the image:

   ```
   docker push <target-repository>:<tag>
   ```

   The AWS console shows the uploaded image URI.
6. Record the image URI in the [Worksheet for AWS hyperscaler](#worksheet-for-aws-hyperscaler).

For AWS instructions, see [Pushing a Docker image](https://docs.aws.amazon.com/AmazonECR/latest/userguide/docker-push-ecr-image.html).
{% endstep %}

{% step %}

### Step 4: Create an S3 bucket for PDI-CLI

Create an S3 bucket for files the container needs at startup.

1. Create an S3 bucket.

   See AWS docs: [Creating a bucket](https://docs.aws.amazon.com/AmazonS3/latest/userguide/create-bucket-overview.html).
2. Record the bucket name in the [Worksheet for AWS hyperscaler](#worksheet-for-aws-hyperscaler).
3. Upload the required directories and files.

   See AWS docs: [Uploading objects](https://docs.aws.amazon.com/AmazonS3/latest/userguide/upload-objects.html).

**S3 bucket directories**

Create these directories in the bucket as needed:

* `root/`
  * Files in this directory are copied to `/home/pentaho/data-integration/data` in the container.
* `jdbc-drivers/`
  * Put JDBC drivers here.
  * Files are copied to Pentaho’s `lib` directory.
* `plugins/`
  * Put additional plugins here.
  * Files are copied to Pentaho’s `plugins` directory.
  * Organize each plugin in its own directory.
* `metastore/`
  * Put metastore content here when jobs require it.
  * Copy your local `.pentaho/` folder into this directory.
  * Content is copied to `/home/pentaho/.pentaho` in the container.

**S3 bucket files**

Upload these files as needed:

* `content-config.properties`
  * Controls which S3 files are copied and where.
  * Add one line per copy instruction:

    ```
    ${KETTLE_HOME_DIR}/<some-dir-or-file>=${APP_DIR}/<some-dir>
    ```
  * Example from the template:

    ```
    ${KETTLE_HOME_DIR}/context.xml=${APP_DIR}/context.xml
    ```
* `content-config.sh`
  * Optional script to configure files, change ownership, install missing apps, and more.
  * Runs after the other files are processed.
    {% endstep %}

{% step %}

### Step 5: Configure and execute PDI-CLI in AWS Batch

Create the AWS Batch resources and run a job using the PDI-CLI image.

Follow AWS guidance at [Getting started with AWS Batch](https://docs.aws.amazon.com/batch/latest/userguide/Batch_GetStarted.html).

1. Create a compute environment.
2. Create a job queue.
3. Create a job definition.

   Set the container image to the ECR image URI from Step 3.
4. Create a job.
5. Set environment variables for your job:
   * `PROJECT_S3_LOCATION`
     * S3 location that contains the project files.
     * Example: `s3://pentaho-samples/`
   * `METASTORE_LOCATION`
     * S3 path to the metastore directory.
     * Content is copied to `/home/pentaho/.pentaho` in the container.
     * Example: `metastore`
   * `PROJECT_STARTUP_JOB`
     * Job (`.kjb`) path to run at startup.
     * Example: `jobs/run_job_write_to_s3/read_csv_from_s3_job.kjb`
   * `LICENSE_TOKEN`
     * License token or license server URL.
     * Example: `http://localhost:7070/license-server/request(Sample)`
   * `PARAMETERS`
     * Parameters passed to the job or transformation.
     * Example: `-param:my_param_name=MYVALUE`

You can now run jobs and transformations using PDI-CLI.
{% endstep %}
{% endstepper %}

### Worksheet for AWS hyperscaler

Use this worksheet to track values during setup:

| Variable                                                       | Record your setting |
| -------------------------------------------------------------- | ------------------- |
| ECR\_IMAGE\_URI (only Platform/PDI Server and Carte Server)    |                     |
| RDS\_HOSTNAME (only Platform/PDI Server and Carte Server)      |                     |
| RDS\_PORT (only Platform/PDI Server and Carte Server)          |                     |
| S3\_BUCKET\_NAME                                               |                     |
| EKS\_CLUSTER\_NAME (only Platform/PDI Server and Carte Server) |                     |
| LICENSE\_TOKEN                                                 |                     |


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter:

```
GET https://docs.pentaho.com/install/legacy-redirects/hyperscalers-legacy/installing-pentaho-on-aws.md?ask=<question>&goal=<endgoal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.
