Get records from stream

This step returns records that were previously generated by another transformation in a job. The records were passed to this step using one of the streaming input steps (for example, Kinesis consumer or Kafka consumer), which are available in the Streaming category of transformations.

This step produces one or more rows and cannot be placed within an existing stream. It must be the first step in a stream because it produces rows.

If you want to add this data to an existing stream, use a join step.

Step name

• Step name: Specifies the unique name of the Get records from stream transformation step on the canvas. You can customize the name or leave it as the default.

Fields

Get Records from Stream Options

Use the Fields table to enter the metadata of the fields you expect from the previous transformation.

Column	Description
Fieldname	Name of the field that contains the records from the previous result.
Type	Type of data. Options include: Number String Date Boolean Integer BigNumber Serializable Binary Timestamp Internet Address
Length	For a field with the type of Number, enter the total number of significant figures in a number. For a field with the type of String, enter the length of the string.
Precision	For a field with the type of Number, enter the number of floating point digits. Otherwise, leave blank.

Note: There is no validation performed on this metadata. It is intended as a design-time aid.

Metadata injection support

This step supports metadata injection. You can use it with ETL metadata injection to pass metadata to your transformation at runtime.

See also

• Kafka consumer