Administering Ambari
Also available as:
PDF
loading table of contents...

Create LZO files

You can output LZO files directly from a Hive query.

  1. Create LZO files as the output of the Hive query.
  2. Use lzo command utility or your custom Java to generate lzo.index for the .lzo files.
    Prefix the query string with these parameters:
    Table 1. Hive LZO Query String Parameters and Values
    Parameter Value
    SET mapreduce.output.fileoutputformat.compress.codec com.hadoop.compression.lzo.LzoCodec
    SET hive.exec.compress.output true
    SET mapreduce.output.fileoutputformat.compress true
    hive -e "SET mapreduce.output.fileoutputformat.compress.codec=com.hadoop.compression.lzo.LzoCodec;SET hive.exec.compress.output=true;SET mapreduce.output.fileoutputformat.compress=true;"