Administration
Also available as:
PDF
loading table of contents...

Pruning Data from Elasticsearch

Elasticsearch provides tooling to prune index data through its Curator utility. For more information about the Curator utility, see Curator Reference.

  1. Use the following command to prune the Elasticsearch data:

    The following is a sample invocation that you can configure through Cron to prune indexes based on the timestamp in the index name.

    /opt/elasticsearch-curator/curator_cli --host localhost delete_indices --filter_list '
         {
           "filtertype": "age",
           "source": "name",
           "timestring": "%Y.%m.%d",
           "unit": "days",
           "unit_count": 10,
           "direction": "older”
         }'

    Using name as the source value causes Curator to look for a timestring value within the index or snapshot name, and to convert that into an epoch timestamp (epoch implies UTC).

  2. For finer-grained control over indexes pruning, provide multiple filters as an array of JSON objects to filter_list. Chaining multiple filters implies logical AND.

    --filter_list '[{"filtertype":"age","source":"creation_date","direction":"older","unit":"days","unit_count":13},
    {"filtertype":"pattern","kind":"prefix","value":"logstash"}]'

For finer-grained control over the indexes that will be pruned, you can also provide multiple filters as an array of JSON objects to filter_list. Chaining multiple filters implies there is an implicit logical AND when chaining multiple filters.

--filter_list '[{"filtertype":"age","source":"creation_date","direction":"older","unit":"days","unit_count":13},
{"filtertype":"pattern","kind":"prefix","value":"logstash"}]'