Kafka Plugin

Overview

Pentaho Kafka Plugin is designed to enable seamless integration with Apache Kafka for streaming and batch data operations. It supports real-time data ingestion and publishing to Kafka topics, making it ideal for scalable, event-driven architectures.

Installation

Pre-Installation

• Ensure that you have access to the Support Portal. Reach out to your Pentaho Support representative if you do not have access.

• Ensure that any Pentaho Data Integration (PDI) tools (pan, kitchen, Carte, Spoon) and Pentaho Server are fully shut down before beginning the installation.

Installation Instructions

1

Download the Plugin

• Visit the Download section in the Pentaho Support portal

  • Download for PDI 10.2

  • Download for PDI 11.0

• Download the zip files:

  • Plugins/Kafka/streaming-kafka-plugin-<version>-<build>-dist.zip

  • Plugins/Kafka/kafka-job-plugins-<version>-<build-dist.zip

2

Extract the files

Unzip both of the dowloaded files

3

Run the Installers (for both unzipped files)

• Execute the appropriate script based on your operating system:

  • install.sh for Linux/macOS

  • install.bat for Windows

• Accept the End User License Agreement (EULA) when prompted

• Choose Installation Path

  • For PDI/Spoon: <PENTAHO_HOME>/data-integration/plugins

  • For Pentaho Server: <PENTAHO_HOME>/pentaho-server/pentaho-solutions/system/kettle/plugins

  • Note: <PENTAHO_HOME> is the absolute path of the folder where you have installed Pentaho

Post-installation Steps

• Restart PDI and/or Pentaho Server after installation.

• Sample transformations are included in the plugin release folder for reference.

Further Details

See Kafka Offset Job Entry, Kafka Consumer Step, Kafka Producer Step.