Fields tab

Parquet Output step

In the Fields tab, you can define properties for the fields being exported. The table below describes each of the options for configuring the field properties.

Property
Description

Parquet Path

Specify the name of the column in the Parquet file.

Name

Specify the name of the PDI field.

Parquet Type

Specify the data type used to store the data in the Parquet file.

Precision

Specify the total number of significant digits in the number (only applies to the Decimal Parquet type). The default value is 20.

Scale

Specify the number of digits after the decimal point (only applies to the Decimal Parquet type). The default value is 10.

Default value

Specify the default value of the field if it is null or empty.

Null

Specify if the field can contain null values.

Note: To avoid a transformation failure, make sure the Default value field contains values for all fields where Null is set to No.

You can define the fields manually, or you can click Get Fields to automatically populate the fields. When the fields are retrieved, a PDI type is converted into an appropriate Parquet type, as shown in the table below. You can also change the selected Parquet type by using the Type drop-down or by entering the type manually.

PDI Type
Parquet Type

InetAddress

UTF8

String

UTF8

TimeStamp

TimestampMillis

Binary

Binary

BigNumber

Decimal

Boolean

Boolean

Date

Date

Integer

Int64

Number

Double

Last updated

Was this helpful?