Scaling Namespaces and Optimizing Data Storage
Also available as:
PDF
loading table of contents...

Use GZipCodec with a one-time job

You can configure GZipcodec to compress the output of a MapReduce job.

To use GzipCodec with a one-time only job, add the options to configure compression for the MapReduce job and configure GZipCodec for the output of the job.
hadoop jar hadoop-examples-1.1.0-SNAPSHOT.jar sort sbr"-Dmapred.compress.map.output=true" 
sbr"-Dmapred.map.output.compression.codec=org.apache.hadoop.io.compress.GzipCodec"
sbr "-Dmapred.output.compress=true" 
sbr"-Dmapred.output.compression.codec=org.apache.hadoop.io.compress.GzipCodec"sbr -outKey org.apache.hadoop.io.Textsbr 
-outValue org.apache.hadoop.io.Text input output