Articles index:
- Introduction (Everyone)
- JSON as logs format (Everyone)
- Logging best practices with Logback (Targetting Java DEVs)
- Logging cutting-edge practices (Targetting Java DEVs)
- Contract first log generator (Targetting Java DEVs)
- ElasticSearch VRR Estimation Strategy (Targetting OPS)
- VRR Java + Logback configuration (Targetting OPS)
- VRR FileBeat configuration (Targetting OPS)
- VRR Logstash configuration and Index templates (Targetting OPS)
- VRR Curator configuration (Targetting OPS)
- Logstash Grok, JSON Filter and JSON Input performance comparison (Targetting OPS)
VRR Curator configuration
Curator configuration
Last piece of the puzzle, Curator will take care of deleting information from ElasticSearch once it has expired according to our policies.There are two ways of deleting information from ElasticSearch using its API
- By deleting documents through a query that uses timestamp -> SLOW, deleting a document requires modifying indices and caches.
- By deleting indices as a whole -> FAST. But you cannot choose what to delete inside an index, it all goes down.
Then curator can delete the oldest indices and keep the newest at the same time.
Finally, Curator is not a daemon, it needs to be executed, normally once a day is enough (really as often as your smaller granularity).
Configuration example:
File available here
---
actions:
1: <- Sets an order of execution
action: delete_indices <- action
description: >-
Delete indices older than 3 days (based on index name), for vrr-*-crit-
prefixed indices. Application/Service name is irrelevant, but this only
applies to "crit" indices. Ignore the error if the filter does not result in an
actionable list of indices (ignore_empty_list) and exit cleanly.
options:
ignore_empty_list: True
timeout_override:
continue_if_exception: False
disable_action: False <- otherwise it wouldn't execute
filters:
- filtertype: pattern
kind: regex
value: vrr-.*-crit- <- Applies to all VRR critical indices, no matter the application. this is only valid if you get an agreement on retention being the same for all application forthe same importance. exclude: - filtertype: age <- Second filter, we are filtering by age source: name direction: older timestring: '%Y-%m-%d' <- date format in the index (set in logstash!) unit: days unit_count: 371 <- delete if older than 371 days ("days" defined in unit, just above) exclude: 2: action: delete_indices description: >- Delete indices older than 2 days (based on index name), for vrr-*-imp- prefixed indices. Application/Service name is irrelevant, but this only applies to "imp" indices. Ignore the error if the filter does not result in an actionable list of indices (ignore_empty_list) and exit cleanly. options: ignore_empty_list: True timeout_override: continue_if_exception: False disable_action: False filters: - filtertype: pattern kind: regex value: vrr-.*-imp-<- Applies to all VRR important indicesexclude: - filtertype: age source: name direction: older timestring: '%Y-%m-%d' unit: days unit_count: 90<- delete if older than 90 daysexclude: 3: action: delete_indices description: >- Delete indices older than 1 days (based on index name), for vrr-*-low- prefixed indices. Application/Service name is irrelevant, but this only applies to "low" indices. Ignore the error if the filter does not result in an actionable list of indices (ignore_empty_list) and exit cleanly. options: ignore_empty_list: True timeout_override: continue_if_exception: False disable_action: False filters: - filtertype: pattern kind: regex value: vrr-.*-low-<- Applies to all VRR low indicesexclude: - filtertype: age source: name direction: older timestring: '%Y-%m-%d' unit: days unit_count: 14<- delete if older than 14 daysexclude:
Next: 11 - Logstash Grok, JSON Filter and JSON Input performance comparison
