MongoDB Output
The MongoDB Output step writes data to a MongoDB collection.
Step name
Step name specifies the unique name of the step on the canvas. You can change it.
Configure the step (tabs)
Configure connection tab
You can configure the connection using either a connection string or individual connection fields.
Connection string
Select Connection String, then enter a connection string URI. Verify the connection by selecting Test Connection.
For connection string formats and options, see the MongoDB documentation.
Common examples:
SSL
mongodb://<hostname>:<port>/?tls=true
SSL and LDAP
mongodb://<username>:<password>@<hostname>:<port>/?tls=true&authSource=$external&authMechanism=PLAIN
LDAP
mongodb://<username>:<password>@<hostname>:<port>/?authSource=$external&authMechanism=PLAIN
SSL and LDAP cluster servers with replicaSet
mongodb://<username>:<password>@<hostname>:<port>/?tls=true&authsource=$external&authMechanism=PLAIN&replicaSet=rs0
SSL and LDAP with replicaSet and readPreference
mongodb://<username>:<password>@<hostname>/?tls=true&authSource=$external&authMechanism=PLAIN&replicaSet=rs0&readPreference=secondary
Kerberos
mongodb://<service-principal>@<hostname>:<port>/?authSource=$external&authMechanism=GSSAPI
Kerberos and SSL
mongodb://<service-principal>@<hostname>:<port>/?authSource=$external&authMechanism=GSSAPI&tls=true
Atlas Cloud/SAAS
mongodb+srv://<username>:<password>@mycluster.qj8y0.mongodb.net/myFirstDatabase?retryWrites=true&w=majority
Configure fields
If you select Configure Fields, specify these values:
Host name(s) or IP address(es)
Host names or IP addresses for MongoDB instances. You can specify ports by using host:port. Separate multiple hosts with commas.
Port
Default port (used if ports are not specified with hosts). Default: 27017.
Enable SSL connection
Connects to a MongoDB server configured for SSL.
Use all replica set members/mongos
Uses all replica sets when multiple hosts are specified. If a replica set has more than one host, the Java driver discovers hosts automatically and tries the next replica set if the selected set is unavailable.
Authentication database
Authentication database.
Username
Username required to access the database. For Kerberos, enter the Kerberos principal.
Password
Password for the username (not required for Kerberos).
Authenticate mechanism
Authentication method. Values include SCRAM-SHA-1, MONGODB-CR, and PLAIN.
Authenticate using Kerberos
Enables Kerberos authentication.
Connection timeout
Connection timeout in milliseconds. Leave blank for no timeout.
Socket timeout
Write operation timeout in milliseconds. Leave blank for no timeout.
Output options tab
Use the Output options tab to control how PDI writes to MongoDB.
If the specified collection does not exist, PDI creates it before inserting documents.
Database
Target database. Select Get DBs to retrieve database names.
Collection
Target collection. Select Get Collections to retrieve collection names. If the collection does not exist, PDI creates it.
Batch insert size
Batch size for bulk insert operations. Default: 100 rows.
Truncate collection
Deletes existing data in the target collection before inserting new data.
Update
Sets the update write method. Upsert and Modifier update require an update field configuration (see Mongo document fields).
Upsert
Changes the write method from insert to upsert. A matched document is replaced based on incoming fields defined in Mongo document fields. If no match exists, PDI inserts a new document.
Multi-update
Updates all matching documents for each update or upsert operation.
Modifier update
Enables modifier operators (for example, $set) to update individual fields within matching documents. To update more than one matching document, select Modifier update and Upsert. Selecting Modifier update, Upsert, and Multi-update applies updates to all matching documents (instead of just the first).
Number of retries for write operations
Number of retry attempts for write operations.
Delay, in seconds, between retry attempts
Seconds to wait between retries.
Write concern (w option)
Minimum number of servers that must confirm the write. Values include: -1 (disable acknowledgement), 0 (disable basic acknowledgement but return socket/network errors), 1 (acknowledge on primary), and values greater than 1 (wait for the specified number of members, including the primary). Select Get custom write concerns to retrieve custom write concerns stored in the repository.
w Timeout
Time (ms) to wait for write acknowledgement. Leave blank for no timeout.
Journaled writes
Waits until mongod acknowledges the write operation and commits data to the journal.
Read preference
Node selection preference: Primary, Primary preferred, Secondary, Secondary preferred, or Nearest. Default is Primary. This option is available when Modifier update is selected.
Mongo document fields tab
Use the Mongo document fields tab to define how incoming fields are written to a MongoDB document.
The Modifier policy column controls when a modifier operation affects a field. This is useful when data for a single MongoDB document is split across multiple incoming rows or when you cannot apply different modifier operations to the same field at the same time.
Name
Incoming field name.
Mongo document path
Hierarchical path (dot notation) to the field in the MongoDB document.
Use field name
Whether to use the incoming field name as the final entry in the path. Values: Y (use incoming field name) or N (do not). When Y, the step assumes a preceding ..
NULL values
Whether to insert null values. Values: Insert NULL or Ignore.
JSON
Indicates the incoming value is a JSON document.
Match field for update
When performing an upsert, PDI matches documents using fields marked Y in this column. The first matching document is updated/replaced (depending on configuration). If no match exists, PDI inserts a new document.
Modifier operation
In-place modifications for existing fields. Supported modifiers include N/A, $set, $inc, $push, and $ (positional operator for arrays).
Modifier policy
When a modifier applies: Insert&Update (default), Insert (only on insert), or Update (only on update).
Get fields
Populates the Name column from incoming fields.
Preview document structure
Shows the JSON structure that will be written to MongoDB.
Example: document structure and field definitions
Input data:
Document field definitions:
first
top1
Y
N
N
N/A
Insert&Update
last
array[0]
Y
N
N
N/A
Insert&Update
address
array[0]
Y
N
N
N/A
Insert&Update
age
array[0]
Y
N
N
N/A
Insert&Update
Resulting structure:
Create/drop indexes tab
Use the Create/drop indexes tab to create or drop MongoDB indexes on one or more fields.
Unless unique indexes are used, MongoDB allows duplicate records to be inserted. Indexing is performed after all rows are processed by the step.
Index fields
Single-field or compound index. For compound indexes, use a comma-separated list of paths. Use dot notation for nested fields. You can specify direction: 1 (ascending) or -1 (descending).
Index op
Whether to create or drop an index.
Unique
Index only unique field values.
Sparse
Index only documents that include the indexed field.
Show indexes
Displays existing indexes.
Example: compound index
The following example shows a compound index for the first and age fields in ascending order:
Metadata injection support
All fields of this step support metadata injection. You can use it with ETL metadata injection to pass metadata to your transformation at runtime.
Last updated
Was this helpful?