Make sure that PDC_WS_DEFAULT_OPS_JOBPOOLMINSIZE and PDC_WS_DEFAULT_OPS_JOBPOOLMAXSIZE have the same value for consistent worker instance management.
| Value | Description |
|---|---|
@yearly (or @annually) | Run once a year, midnight, Jan 1st |
@monthly | Run once a month, midnight, first of the month |
@weekly | Run once a week, midnight between Sat/Sun |
@daily (or @midnight) | Run once a day, midnight |
@hourly | Run once an hour, the beginning of the hour |
@every <number>m | Run at a custom interval, where <number> specifies the number of minutes. For example, @every 5m runs the job every 5 minutes. |
Users without the configured roles receive standard (PDO-based) chatbot responses.
The profiles value should match with the profile name defined for COMPOSE_PROFILES parameter in vendor/env.default (for example, core).
The profiles value used to disable the chatbot must not match any active profile defined for COMPOSE_PROFILES parameter in vendor/env.default (for example, core). If an admin later updates the profile's value, a name that is listed in vendor/env.default , the chatbot frontend and backend services start automatically.
| Environment variable | Description |
|---|---|
PDC_CHATBOT_API_KEY | API key used to authenticate with the configured LLM provider. The chatbot backend does not start if this value is missing or invalid. |
PDC_CHATBOT_LLM | Name of the LLM used to generate chatbot responses. |
PDC_CHATBOT_LLM_BASE_URL | Base URL of the LLM provider endpoint. Use this value for self-hosted or non-default LLM providers. |
PDC_CHATBOT_EMBEDDING_MODEL | Embedding model used to generate vector representations for conversational search. You cannot change this value after the initial setup. |
PDC_CHATBOT_TARGET_VECTOR_DIMENSIONS | Number of dimensions produced by the embedding model. This value must match the dimensions supported by the embedding model. |
PDC_CHATBOT_TARGET_VECTOR_BATCH_SIZE | Number of records processed in a single batch during vector indexing. Higher values improve throughput but increase memory usage. |
PDC_CHATBOT_TARGET_VECTOR_MAX_CONCURRENT_BATCHES | Maximum number of vector indexing batches processed in parallel. Increasing this value increases the load on the system. |
PDC_CHATBOT_CONVERSATIONAL_ROLES | User roles allowed to receive conversational (vector-based) chatbot responses. You can add additional roles separated by a comma. |
PDC_CHATBOT_SCHEDULE_VECTOR_INDEX_UPDATE | Schedule that controls how often the chatbot refreshes the vector index (for example, @daily). |
You cannot change the embedding model after the initial setup. If you need to use a different embedding model, you must redeploy the chatbot with a fresh vector index.
| YAML parameter | Description |
|---|---|
apiKey | API key used to authenticate with the configured LLM provider. The chatbot backend does not start if this value is missing or invalid. |
model | Name of the LLM used to generate chatbot responses. |
baseUrl | Base URL of the LLM provider endpoint. Required for self-hosted or custom LLM providers. |
embeddingModel | Embedding model used to generate vector representations for conversational search. You cannot change this value after the initial setup. |
targetVectorDimensions | Number of dimensions produced by the embedding model. This value must match the dimensions supported by the embedding model. |
targetVectorBatchSize | Number of records processed in a single batch during vector indexing. |
targetVectorMaxConcurrentBatches | Maximum number of vector indexing batches processed in parallel. |
conversationalRoles | User roles allowed to receive conversational (vector-based) chatbot responses. You can add additional roles separated by a comma. |
schedule | Cron expression that controls how often the chatbot refreshes the vector index (for example, 0 0 * * *). |
By default, conversational search is enabled only for the Data Steward role. To know more about user roles in Data Catalog, see /spaces/RAKLVv06oBKpy9VLbw7P/pages/Oq6KhWb59ZQX4aRDW8gs.
| Field | Description |
|---|---|
| Host | IP address or domain name of SMTP server |
| Port | Port number for SMTP server |
| Username | User name in *<mail userID>*@*<domain>*.com format |
| Password | Password for user name specified above |
| Sender Mail | Sender mail ID in *<mail userID>*@*<domain>*.com format |
| Encryption |
|
| Parameter | Description |
|---|---|
<PDC_HOST> | The host name or IP address of your Data Catalog instance. |
<TENANT_NAME> | The tenant name, typically "pdc". |
<TOKEN_VALUE> | A valid authentication token (must be obtained through IAM authentication). |
<SMTP_PASSWORD> | The password for the SMTP server authentication. |
<REPLY_TO_DISPLAY_NAME> | The display name for the reply-to email address. |
<SMTP_PORT> | The port number used by the SMTP server. |
<SMTP_HOST> | The SMTP server host address). |
<REPLY_TO_EMAIL> | The reply-to email address. |
<FROM_EMAIL> | The email address used to send notifications. |
<FROM_DISPLAY_NAME> | The display name associated with the sender’s email. |
<ENVELOPE_FROM> | The envelope sender address (optional). |
<SMTP_USERNAME> | The username for SMTP authentication. |
| Parameter | Value | Description |
|---|---|---|
| minReplicas | 1 | Minimum number of Job Server pods |
| maxReplicas | 100 | Maximum number of Job Server pods |
| targetCPUUtilizationPercentage | 80 | Scale-out threshold for CPU usage |
| targetMemoryUtilizationPercentage | 80 | Scale-out threshold for memory usage |
- name: USE_EASYOCR
value: "true"
To use a custom model, set ML_USE_LLM_FOR_CONTENT_PROCESSING=false.
To use a custom model, set useLLMForContentProcessing=false.
| Parameter | Description | Example |
|---|---|---|
id | Unique identifier (UUID) for the ML server. | 916d3b20-7fd6-49d2-b911-cc051f56e837 |
name | Display name for the server. This name appears in the UI. | MLflowServer |
type | Type of server (enum value). For ML server, use ‘MlFlow’. | |
URL | The base URL of the ML server. | http://mlflow.mycompany.com |
config | Configuration keys specific to ML server you are configuring. Include either, only if authentication is enabled:- Username and password
|
| Parameter | Description | Example |
|---|---|---|
id | Unique identifier (UUID) for the Triton server. | 44e2fa51-e3af-4094-8dd3-c62320952de5 |
name | Display name for the server. This name appears in the UI. | Triton-Prod-server |
type | Type of server (enum value). For Triton server, use ‘Triton’. | Triton |
URL | The base URL where the Triton server is deployed. | http://192.168.0.10 |
config.metadata_port | HTTP port configured when deploying the Triton server.
The default port is 8000. | 8000 |
config.metrics_port | Metrics port configured by the user when deploying the Triton server.
The default is 8002. | 8002 |
| Parameter | Description | Example |
|---|---|---|
id | The site ID (unique identifier) of the Tableau site to connect to, as seen in the Tableau Cloud URL. | dev-8f012f9ca7 |
name | Display name for the server. This name appears in the UI. | TableauServer |
type | Type of server (enum value). For Tableau server, use ‘Tableau’. | Tableau |
URL | The Tableau REST API authentication endpoint. Use the signin endpoint for the Tableau site. | https://prod-apnortheast-a.online.tableau.com/api/3.22/auth/signin |
config | Configuration keys specific to the Tableau server you are configuring. | |
- pat_name | The name of the Tableau Personal Access Token (PAT) used for authentication. | |
- pat_secret | The secret key associated with the PAT. Ensure this is stored securely and never exposed. |
| Permission | Description |
|---|---|
App.Read.All | View all Power BI apps |
Capacity.Read.All | View all capacities |
Dashboard.Read.All | Read dashboards |
Dataflow.Read.All | Read dataflows |
Dataset.Read.All | View all datasets |
Report.Read.All | Read reports |
Tenant.Read.All | View all content in tenant |
Workspace.Read.All | View all workspaces |
| Field | Description |
|---|---|
| Client ID | From Azure AD App registration |
| Client Secret | Securely generated during Azure AD registration |
| Tenant ID | Azure AD Directory unique identifier |
| OAuth2 Token URL | https://login.microsoftonline.com/{tenant}/oauth2/v2.0/token |
| API Host URL | https://api.powerbi.com |
| Username | Credentials for the Power BI server. |
| Password | Credentials for the Power BI server. |
/opt/pentaho/pdc-docker-deployment/conf/external-data-source
servers:
- id: <server-uuid>
name: PowerBIServer
type: PowerBI
url: https://login.microsoftonline.com/organizations/oauth2/v2.0/token
config:
client_id: <your-client-id>
username: <your-username>
password: <your-password>
client_secret: <your-client-secret>
host_url: https://api.powerbi.com
| Parameter | Description | Example |
|---|---|---|
id | The unique identifier (UUID) for the Power BI server connection. | 7f3a9c2e-4b6d-4f1a-9c83-2e8b7d6a4f21 |
name | Display name for the server. This name appears in the Data Catalog UI. | PowerBIServer |
type | Type of server (enum value). For Power BI server, use ‘PowerBI’. | PowerBI |
URL | The Microsoft identity platform OAuth 2.0 token endpoint. Used for authentication requests. | https://prod-apnortheast-a.online.tableau.com/api/3.22/auth/signin |
config | Configuration keys specific to the Power BI server you are configuring. | |
- client_id | The application (client) ID generated during Azure AD app registration. | 12345678-abcd-1234-abcd-1234567890ab |
- username | The Microsoft account username with access to the Power BI service. | admin@contoso.com |
-password | The password associated with the Microsoft account username. Ensure this is stored securely. | |
-client_secret | The client secret generated for the Azure AD app. Ensure this is stored securely and never exposed. | abcdEFGH12345!@#xyz |
-tenant_id | The Azure Active Directory tenant ID used for Service Principal authentication. | contoso.onmicrosoft.com |
-host_url | The base API endpoint for Power BI service. | https://api.powerbi.com |
version: 0.0.1
consumers:
console:
file:
- path: /path/to/file
http:
- name: Marquez
url: http://localhost:5001
- name: PDC
url: https://pdc.example.com
endpoint: /lineage/api/events
authenticationParameters:
endpoint: /keycloak/realms/pdc/protocol/openid-connect/token
username: user
password: pass
client_id: pdc-client-in-keycloak
scope: openid
| Field | Value |
|---|---|
| Vendor | Active Directory |
| Connection URL | ldap://*<LDAP\_SERVER>*:*<PORT>* such as: ldap://localhost:389 |
| Field | Value |
|---|---|
| Bind type | Select simple |
| Bind DN | DN for your LDAP admin user, such as: cn=administrator,dc=example,dc=com |
| Bind credentials | Password for the LDAP admin user |
| Field | Value |
|---|---|
| UI display name | Name to display, such as LDAPS |
| Vendor | Select Active Directory |
| Connection URL | ldaps://*<LDAP\_SERVER>*:*<PORT>*such as: ldaps://ldap.example.com:636 |
| Field | Value |
|---|---|
| Bind type | Select simple |
| Bind DN | DN to bind to the LDAP server, such as: cn=admin,dc=example,dc=com. |
| Bind credentials | password for the Bind DN |
| Field | Value |
|---|---|
| Edit mode | It is a best practice to set this to Readonly |
| Users DN | Specify the DN where the user entries are located, such as: ou=users,dc=example,dc=com |
| Username LDAP attribute | cn |
| RDN LDAP attribute | cn |
| UUID LDAP attribute | objectGUID |
| User object classes | person, organizationalPerson, user |
| Field | Value |
|---|---|
| Name | Business_User_Role_Mapper_To_LDAP_USERS |
| Mapper type | hardcoded-ldap-role-mapper |
| Role | Business_User (select from menu and click Assign) |
| Field | Value |
|---|---|
| Name | Data_User_Role_Mapper_To_LDAP_USERS |
| Mapper type | hardcoded-ldap-role-mapper |
| Role | Data_User (select from menu and click Assign) |
| Field | Value |
|---|---|
| Sign-in method | OIDC – OpenID Connect |
| Application type | Web Application |
| App integration name | CatalogPlus_10.2.1 |
| Grant type | Authorization Code |
| Sign-in redirect URIs | Keycloak Identity Provider Redirect URI copied in the Add OpenID Connect provider in Keycloak task |
| Field | Value |
|---|---|
| Controlled access | Select the default value, Allow everyone in your organization to access |
| Enable immediate access | Clear the checkbox |
The token expires based on the expires_in value in the response. If the token expires, generate a new token and retry the API call.
You have an admin access token to use as Authorization: Bearer <access-token> in Keycloak Admin API requests.
If your IdP administrator requests the certificate, provide the first value in x5c[]. This is the base64-encoded X.509 certificate.
firstBrokerLoginFlowAlias and postBrokerLoginFlowAlias values as shown unless your Keycloak administrator has configured different broker flows for Data Catalog.backchannelSupported is enabled, Keycloak uses a server-to-server logout flow: Keycloak pod → PingFederate. If this request times out, it typically indicates a network connectivity issue between the Keycloak pod and PingFederate (for example, firewall rules, DNS resolution, routing, or a blocked port).Tip: If you copy the X509Certificate value from XML, remove line breaks and extra spaces so the certificate is stored as a single continuous base64 string.
Important: Create one role mapper per Data Catalog role. If you do not create a mapper for a role, users will not receive that role through the IdP.
Replace the placeholder values in angle brackets (< >) with your actual Jira credentials, project details, and proxy details if your environment uses a forward proxy.