Alteryx lineage
Alteryx is an ETL and analytics platform that uses visual workflows to read data from one or more sources, apply a sequence of processing tools, and write the results to one or more targets. These tools can filter, join, group, select, rename, calculate, validate, or otherwise transform the data.
When an Alteryx workflow is imported, Pentaho Data Catalog analyzes the workflow definition and extracts lineage information about its source assets, target assets, transformations, connections, and column relationships. Data Catalog matches the workflow inputs and outputs to the corresponding cataloged assets and displays the resulting relationships in the Lineage view. When the output of one imported workflow is used as the input to another workflow, Data Catalog can connect the workflows to display an end-to-end, cross-workflow lineage path.
In the lineage graph, the imported Alteryx workflow appears as a Job between the source and target assets. Select the Job to review its properties and the transformation metadata captured from the workflow definition. For supported Alteryx tools, Data Catalog also displays the available tool-specific transformation logic. For structured outputs, you can also expand an output column to identify its upstream input columns and review the transformations that affect that column.
Alteryx lineage represents design-time lineage. It describes how the imported workflow is configured to process data. It does not confirm that a particular workflow execution ran or completed successfully.
Use Alteryx lineage to:
Trace an Alteryx workflow output to its upstream sources.
Follow data across multiple connected Alteryx workflows.
Identify the transformations applied between source and target assets.
Understand how an output column is derived from one or more input columns.
Assess the potential downstream effect of changes to source tables or columns.
Support governance, audit, troubleshooting, and data-quality investigations.
How Alteryx lineage works in Data Catalog
Data Catalog creates Alteryx lineage by analyzing imported Alteryx workflow definitions and matching the workflow inputs and outputs to assets in the catalog. A PDC administrator can import workflows directly from a configured Alteryx environment or provide workflow files through an offline import. Both methods create the same type of lineage in Data Catalog. To know more, see Configure Alteryx lineage.
The lineage process consists of the following stages:
An administrator imports one or more Alteryx workflow definitions from a connected Alteryx environment or from workflow files into Data Catalog.
Data Catalog parses and analyzes each workflow definition.
Data Catalog identifies the source and target connections, participating assets, Alteryx tools (represented as transformations in Data Catalog), transformation order, available tool-specific logic, and available table-level and column relationships.
Data Catalog maps workflow inputs and outputs to the corresponding cataloged data assets. If Data Catalog cannot match a relational input or output to an existing asset, it can create a representative entity for lineage purposes.
For the most accurate asset matching, add the participating data sources and ingest their metadata before importing the workflows.
Data Catalog represents the imported workflow as a Job between the source and target assets in the lineage graph. The Job contains the transformation metadata and the available input, output, and column relationships extracted from the workflow.
When the output of one imported workflow is used as the input to another workflow, Data Catalog connects the Jobs to display cross-workflow lineage. This provides an end-to-end view of data flow across multiple Alteryx workflows.
For Alteryx tools with detailed support, Data Catalog displays the available transformation logic or configuration. For other tools, Data Catalog can display the transformation name, identifier, and order, but detailed tool-specific logic might not be available.
Alteryx assets and relationships represented in lineage
The Alteryx lineage graph represents the source and target connections and assets used by an imported workflow and the Job that connects them. Select or expand an element in the graph to review its associated assets, columns, transformations, and relationships. When multiple imported workflows use the same intermediate asset, Data Catalog can connect their Jobs to display cross-workflow lineage.
The following image displays the Alteryx assets and relationships represented in lineage in Data Catalog:
1. Source and target connections
The source and target cards identify the data connections used by the Alteryx workflow. For example, a workflow might read data from an Oracle connection and write the processed data to a Snowflake connection. A connection can contain one or more assets, such as database tables or files, that participate in the lineage. When the participating data connections and assets are already available in Data Catalog, the imported lineage is associated with those cataloged assets. If Data Catalog cannot resolve a relational input or output to an existing cataloged asset, it can create representative connections and assets for lineage purposes.
Representative assets identify the objects referenced by the workflow but do not contain the complete metadata available from a standard metadata ingest.
2. Source and target assets
Source assets are the tables, files, or other data objects that the Alteryx workflow reads. Target assets are the data objects to which the workflow writes its processed results. In the lineage graph, source assets appear upstream of the Alteryx Job, and target assets appear downstream of the Job.
For example:
Oracle source table → Alteryx Job → Snowflake target table
These relationships provide table-level lineage between the workflow inputs and outputs. Expand an asset to view the columns that participate in the lineage.
3. Alteryx Job
Data Catalog represents an imported Alteryx workflow as a Job node between its source and target assets. The Job retains the name of the imported workflow and provides information about how the workflow is represented in lineage. When you select the Job node, you can review its details, transformations, outputs, and column relationships. Depending on the available metadata, the Job details can include:
Job name
Namespace
Processing type
Integration
Job type
For example, an Alteryx workflow can appear as a batch transformation Job associated with the Alteryx Server integration.
The Job represents the imported design-time workflow definition. It does not represent or confirm the status of a specific workflow execution.
4. Transformations
Transformations represent supported Alteryx tools that process data as it moves from the source assets to the target assets. Alteryx refers to these workflow components as tools. Data Catalog represents the tools as transformations. These tools can filter, group, join, select, rename, validate, calculate, summarize, or otherwise modify the data. Transformation types can include such as Filter, Formula, Join, Select, Summarize, Unique, and so on.
Data Catalog displays transformations in the Transformations section of the Job details rather than as separate nodes in the lineage graph. For each captured Alteryx tool, Data Catalog can display:
Transformation name
Transformation or tool identifier
Transformation category
Sequence
For supported Alteryx tools, Data Catalog can also display tool-specific logic or configuration, such as:
Filter expression
Grouping fields
Selected or renamed fields
Other tool-specific configuration details
For tools without detailed logic support, the transformation name, identifier, and processing order can still appear, but the tool-specific definition might not be available.
The transformation sequence indicates the order in which the captured operations occur in the workflow. The number appended to a transformation name, such as Filter-2, identifies the Alteryx tool instance and does not necessarily indicate its processing order.
5. Outputs, inputs, and columns
The Outputs section of the Job details lists the target assets and output columns produced by the workflow. You can expand an output column to review:
The upstream source connection
The source asset
The input column or columns
The transformations associated with the selected output column
The available definition or configuration for supported transformations
For example, an output column named AC_Officer_Code can be traced to an upstream Oracle column named MIS_CODE and to the transformations applied between the source and target.
Column-level lineage helps you determine how an output column is derived and which upstream assets and transformations can affect it. Different output columns can have different source-column mappings and transformation paths. The amount of column-level detail available depends on the workflow design, the Alteryx tools, the level of transformation support, and the metadata available for the source and target assets.
6. Cross-workflow relationships
Cross-workflow lineage shows how data flows through multiple imported Alteryx workflows. When the output asset of one workflow is used as the input asset of another workflow, Data Catalog connects the corresponding Jobs through the shared asset.
For example, Workflow A extracts sales data from an Oracle source table and writes the prepared data to a staging table. Workflow B reads the staging table, applies additional transformations, and loads the results into a Snowflake reporting table.
Use the upstream and downstream controls in the Lineage view to follow the data path across the connected workflows. Cross-workflow lineage helps you identify dependencies and assess the potential effect of a change beyond a single Alteryx workflow.
View Alteryx lineage
View Alteryx lineage to identify the source and target assets associated with an imported workflow, review the workflow transformations, and trace output columns to their upstream inputs.
Perform the following steps to view the lineage associated with an Alteryx workflow:
Prerequisites
Before you view Alteryx lineage, ensure that the following requirements are met:
An administrator configured the Alteryx lineage integration and successfully imported the required workflows. For details, see Configure an Alteryx server connection in Data Catalog and Import and sync Alteryx workflows from Alteryx Server in Data Catalog. If integration is unavailable, you can Import Alteryx workflows in offline mode.
For the most accurate asset matching, the source and target data sources used by the workflows have been added to Data Catalog, and their metadata has been ingested.
If Data Catalog cannot resolve a relational workflow input or output to an existing cataloged asset, it can display a representative connection and asset for lineage purposes. Representative assets might not contain the complete metadata available from a standard metadata ingest.
Procedure
In the left navigation menu, click Data Canvas. The list of data sources added appears.
Navigate to a source or target asset used by the Alteryx workflow, then click the Summary tab if it is not already displayed.
On the Lineage pane, click View Lineage.
The Lineage page displays the available upstream and downstream relationships for the selected asset.
Review the source and target elements in the lineage graph. The graph can include:
Source and target connections
Source and target assets
Columns that participate in the lineage
An Alteryx Job between the source and target assets
Intermediate assets and additional Jobs in a cross-workflow lineage path
Expand a source or target asset to view the columns that participate in the lineage. The connection appears as the outer card. The participating table or other data asset appears inside the connection card, along with its columns. You can also use the upstream and downstream controls to adjust the number of relationships displayed in the lineage graph.
Use the controls on the Lineage page as needed:
Search: Find an asset or Job in the displayed graph.
Upstream: Increase or decrease the number of upstream relationship levels.
Downstream: Increase or decrease the number of downstream relationship levels.
Zoom in and Zoom out: Adjust the graph scale.
Reset view: Return the graph to its default scale and position.
Select a source or target asset to open the Details panel.
Depending on the selected asset and the metadata available to you, the panel can display asset properties and governance information.
Select the Alteryx Job node between the source and target assets.
The Details panel displays information about the imported workflow. To know more, see Alteryx assets and relationships represented in lineage.
Review the Job and Job Type sections.
Depending on the metadata available for the workflow, the details can include:
Job name
Namespace
Processing Type, such as BATCH
Integration, such as Alteryx Server
Job Type, such as transformation
The Job represents the imported design-time workflow definition. It does not indicate whether a specific workflow execution ran or completed successfully.
Expand Transformations to review the Alteryx tools captured from the workflow.
Alteryx refers to workflow processing components as tools. Data Catalog represents these tools as transformations. For each captured transformation, Data Catalog can display:
Transformation name
Tool or transformation identifier
Category
Sequence
For supported Alteryx tools, Data Catalog can also display tool-specific logic or configuration, such as filter expressions, grouping fields, and selected or renamed fields. For other tools, the name, identifier, and order can appear even when detailed logic is not available.
Use the sequence value to understand the processing order. The number appended to a transformation name, such as Filter-2, identifies the tool instance and does not necessarily indicate its position in the workflow.
Expand Outputs.
The section displays the target assets and output columns produced by the selected Job.
Expand a target asset, and then expand the output column that you want to investigate.
Under Inputs, review the upstream information associated with the selected output column.
Depending on the workflow, the information can include:
Source connection
Source asset
Input column or columns
Grouping or mapping information
Review the transformations associated with the selected output column.
For supported transformations, Data Catalog displays the available definition or configuration. The column-level transformation path can include operations that directly modify or rename the column and operations that affect which rows reach the output, such as filters or uniqueness checks.
Select another output column to review its input columns and transformation path.
Each output column can have a different source mapping and set of transformations.
To follow lineage across multiple Alteryx workflows, increase the upstream or downstream value.
When the output asset of one imported workflow is used as the input asset of another workflow, the lineage graph connects the corresponding Jobs through the shared asset.
For example:
Source table → Alteryx Job 1 → Intermediate table → Alteryx Job 2 → Target table
Result
You have opened the Alteryx lineage for the selected asset. You can identify the Alteryx Job that connects the source and target assets, review the transformations captured from the workflow, trace output columns to their upstream inputs, and follow relationships across connected Alteryx workflows.
Last updated
Was this helpful?