Metadata rules
Metadata rules execute rule definitions on a selected data source.
On the left navigation menu, click Data Operations. On the Rules on the card, click Metadata Rules to open the Manage Metadata Rules page.
The information in the Metadata Rules table is described below.
Name
The name of the metadata rule.
Datasource
The name of the data source.
Owner
The creator of the metadata rule.
Last Run
The date and time of the last execution of the metadata rule.
Scheduled
The scheduled time for the metadata rule’s execution if applied.
Details
Click View to open the overview and history of the metadata rule:
The Overview tab lists the metadata rule name, data source asset, and rule definition.
The History tab lists previous rule executions including the data source, start time, elapsed time, and the status, The tab allows you to select Terminate to kill the rule being executed, or click Statistics to view the success and failure details of the operation.
The More options icon:
Click and choose Run to run the metadata rule.
Click and choose Edit to edit the metadata rule.
Click and choose Delete to delete the metadata rule.
The Run icon. Click to execute the selected metadata rule.
The Delete icon. Click to delete the selected metadata rule.
Click Add Rule to add a new metadata rule. See the following topics when working with metadata rules:
Create a metadata rule
You create metadata rules to execute rule definitions on a selected data source.
Perform the following steps to create a new metadata rule:
For large data volumes, such as more than 10,000 matching results, do not use a single metadata rule to run data discovery or profiling, and metadata update actions on the same data source. This configuration can cause metadata conflicts and rule failure. To prevent this issue, create two separate rule definitions and rules: one for data discovery or profiling and another for metadata update actions, such as applying tags or business terms. If you trigger both rules at the same time, the Rules Engine queues them and processes them sequentially for the same data source.
If you create a rule with a rule definition that contains a Delete operation, you get a message that since a Delete operation is configured, all other configured operations will be ignored.
In the left navigation menu, click Data Operations.
The Manage Data Operations page opens.
On the Metadata Rules card, click Add New, and then select Add Rule from the menu.
The Add Rule page opens.
Enter the Rule Name and Description (optional) and Click Create and Configure.
In Select Rule Definition, select the rule definition if you have already created it, or click New Definition to create a rule definition. See Create a rule definition for more information.
(Applicable for entities) Click Add Scope and select one or more Source Data Assets current rule can be executed.
If the rule definition you selected has Migrate or Move as a process, you see an additional field, Destination Data Asset.
(Optional) Select the Destination Data Asset.
You can select a destination data source that is:
not the same as the Source data source
supported for migration by Data Optimizer and enabled for migration (the Available for Migration setting for the data source is Enabled)
If you use an HDFS data source as a source, the Destination Data Source field does not appear, because the destination is internally configured in the HDFS cluster.
(Optional) Select the Save and Run checkbox to save and execute the new rule definition.
(Optional) Select Add Schedule to set a frequency or a specific time and day to run the metadata rule.
Select the Save to save the new rule definition without executing.
Since multiple rules can run simultaneously, if a rule is already running on the Source data source, any additional rule intended for the same data source waits until the running rule finishes. The waiting rule remains in the CREATED state until the running rule finishes, after which the waiting rule can start.
The metadata rule is created, and a confirmation message appears on the page.
To review a metadata rule's progress, go to Data Operations → Metadata Rules, select the desired rule, then click View and open the History tab.
Running a metadata rule
Running a metadata rule executes its associated rule definition. You can run a metadata rule immediately after creation, if necessary.
Perform the following steps to run a metadata rule:
In the left navigation menu, click Data Operations.
The Manage Data Operations page opens.
On the Metadata Rules card, click Rules.
The Manage Metadata Rules page opens.
Locate the name of the rule that you want to run and select its check box.
Click Run.
The metadata rule runs after any current operations complete.
To check the rule’s status, click View and select the History tab.
Edit a metadata rule
You can edit an existing metadata rule if needed, to change the rule definition or data source on which it runs.
Perform the following steps to edit a rule:
In the left navigation menu, click Data Operations. The Manage Data Operations page opens.
On the Metadata Rules card, click Rules. The Manage Metadata Rules page opens.
Locate the name of the rule that you want to edit, then in its row, click More options (
) and select Edit.
The metadata rule opens.Edit the fields as needed.
Click Apply.
The metadata rule is saved with your changes, and a status message appears in the upper corner of the page.
Delete a metadata rule
You can delete a metadata rule if it is no longer needed.
Perform the following steps to delete a metadata rule:
In the left navigation menu, click Data Operations.
The Manage Data Operations page opens.
On the Metadata Rules card, click Rules.
The Manage Metadata Rules page opens.
Locate the name of the metadata rule that you want to delete, then in its row, click More options (
) and select its check box.Click Delete.
The metadata rule is deleted, and a confirmation message appears in the upper corner of the page.
Last updated
Was this helpful?