# Unique Rows

The **Unique Rows** step removes duplicate rows from the input stream and outputs only unique rows.

### Prerequisites

The input stream must be sorted before the Unique Rows step. Otherwise, only consecutive duplicate rows are correctly analyzed and filtered.

If you cannot sort the input stream, use [Unique Rows (HashSet)](/pdia-data-integration/pdi-transformation-steps-reference-overview/unique-rows-hashset.md).

### Step name

* **Step name**: Specify the unique name of the step on the canvas. You can customize the name or leave it as the default.

### Settings

| Option                      | Description                                                                                                                                                                                                                             |
| --------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **Add counter to output?**  | Select to add a counter field to the output stream.                                                                                                                                                                                     |
| **Counter field**           | Name of the counter field.                                                                                                                                                                                                              |
| **Redirect duplicate row**  | Select to send duplicate rows to the step error stream. If cleared, duplicates are discarded.                                                                                                                                           |
| **Error description**       | Error message used when redirecting duplicate rows. Available only when **Redirect duplicate row** is selected.                                                                                                                         |
| **Fields to compare table** | <p>Fields used to determine uniqueness.</p><p>Select <strong>Get</strong> to insert all incoming fields.</p><p>If you do not specify any fields, the step compares the entire row.</p>                                                  |
| **Ignore case**             | <p>Set to <code>Y</code> to ignore case when comparing strings.</p><p>For example, <code>Kettle</code>, <code>KETTLE</code>, and <code>kettle</code> are treated as equivalent when <strong>Ignore case</strong> is <code>Y</code>.</p> |

### See also

* [Unique Rows (HashSet)](/pdia-data-integration/pdi-transformation-steps-reference-overview/unique-rows-hashset.md)


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://docs.pentaho.com/pdia-data-integration/pdi-transformation-steps-reference-overview/unique-rows.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.