Verify or edit core-site XML file

Note: If you plan to run MapReduce jobs on an Amazon EMR cluster, make sure you have read, write, and execute access to the S3 Buffer directories specified in the core-site.xml file on the EMR cluster.

You must edit the core-site.xml file to add information about your AWS Access Key ID, your Access key, and your LZO compression setting.

Perform the following steps to edit your core-site.xml:

  1. Navigate to the location where you have saved the previously edited site configuration files and open the core-site.xml file.

  2. Add the following values:

Parameter

Values

fs.s3.awsAccessKeyId

​Value of your S3 AWS Access Key ID.```xml fs.s3.awsAccessKeyId [INSERT YOUR VALUE HERE]

3. If needed, enter the AWS Access Key ID and Access Key for S3N like this:

Parameter

Values

fs.s3n.awsAccessKeyId

Value of your S3N AWS Access Key ID.```xmlfs.s3n.awsAccessKeyId[INSERT YOUR VALUE HERE]

4. Add the following values:

Parameter

Values

fs.s3n.impl

fs.s3.impl

fs.s3a.impl

5. LZO is a compression format that Amazon EMR supports. If you want to configure for LZO compression, you need to download a JAR file. If you do not, you need to remove a parameter from the `core-site.xml` file.

-   If you are not using LZO compression, remove any references to the **iocompression** parameter in the `core-site.xml` file: **com.hadoop.compression.lzo.LzoCodec**
-   If you are not using LZO compression, download the LZO JAR and add it to `pentaho-big-data-plugin/hadoop-configurations/emr3x/lib` directory. The LZO JAR can be found here: [http://maven.twttr.com/com/hadoop/gplcompression/hadoop-lzo/0.4.19/](http://maven.twttr.com/com/hadoop/gplcompression/hadoop-lzo/0.4.19/).

6. Save and close the file.

Note: You must edit the named connection and upload the edited site configuration file in order for this change to take effect.

Last updated

Was this helpful?