Skip to main content
The Retention Rules API allows you to define and manage data retention rules across datasources in Apache Druid. Retention rules determine what data is retained in the cluster through load, drop, and broadcast rules.

​
Update retention rules for a datasource

Updates one or more retention rules for a datasource. This request overwrites any existing rules.
Rules are read in order. The first matching rule is applied to a segment.
curl "http://ROUTER_IP:ROUTER_PORT/druid/coordinator/v1/rules/kttm1" \
--header 'X-Druid-Author: doc intern' \
--header 'X-Druid-Comment: submitted via api' \
--header 'Content-Type: application/json' \
--data '[
    {
        "type": "broadcastForever"
    },
    {
        "type": "loadForever",
        "tieredReplicants": {
            "_default_tier": 2
        },
        "useDefaultTierForNull": true
    },
    {
        "type": "dropByPeriod",
        "period": "P1M"
    }
]'
​
dataSource
string
required
Name of the datasource.
​
X-Druid-Author
string
Author of the configuration change for audit history.
​
X-Druid-Comment
string
Description of the update for audit history.
​
rules
array
required
Array of retention rule objects. Each rule must have a type field.
{}

​
Update default retention rules

Updates default retention rules that apply to all datasources. Submit an empty array to remove all default rules. 
curl "http://ROUTER_IP:ROUTER_PORT/druid/coordinator/v1/rules/_default" \
--header 'Content-Type: application/json' \
--data '[
    {
        "type": "loadByInterval",
        "tieredReplicants": {},
        "useDefaultTierForNull": false,
        "interval": "2010-01-01/2020-01-01"
    }
]'
​
X-Druid-Author
string
Author of the configuration change.
​
X-Druid-Comment
string
Description of the update.
​
rules
array
required
Array of default retention rule objects.
{}

​
Get all retention rules

Retrieves all current retention rules in the cluster including the default rule. 
curl "http://ROUTER_IP:ROUTER_PORT/druid/coordinator/v1/rules"

{
    "_default": [
        {
            "tieredReplicants": {
                "_default_tier": 2
            },
            "type": "loadForever"
        }
    ],
    "social_media": [
        {
            "interval": "2023-01-01T00:00:00.000Z/2023-02-01T00:00:00.000Z",
            "type": "dropByInterval"
        }
    ],
    "wikipedia_api": []
}

​
Get retention rules for a datasource

Retrieves retention rules for a single datasource. 
curl "http://ROUTER_IP:ROUTER_PORT/druid/coordinator/v1/rules/social_media?full=null"
​
dataSource
string
required
Name of the datasource.
​
full
boolean
Include default retention rules in the response.
[
    {
        "interval": "2020-01-01T00:00:00.000Z/2022-02-01T00:00:00.000Z",
        "type": "dropByInterval"
    },
    {
        "interval": "2010-01-01T00:00:00.000Z/2020-01-01T00:00:00.000Z",
        "tieredReplicants": {
            "_default_tier": 2
        },
        "type": "loadByInterval"
    },
    {
        "tieredReplicants": {
            "_default_tier": 2
        },
        "type": "loadForever"
    }
]

​
Get audit history for all datasources

Retrieves the audit history of rules for all datasources over a time period. 
curl "http://ROUTER_IP:ROUTER_PORT/druid/coordinator/v1/rules/history?interval=2023-07-13%2F2023-07-19"
​
interval
string
ISO-8601 time interval delimited by / (e.g., 2023-07-13/2023-07-19).
​
count
integer
Limit the number of results to the last N entries.
​
key
string
Datasource name.
​
type
string
Type of audit entry (typically “rules”).
​
auditInfo
object
Information about who made the change including author, comment, and IP address.
​
payload
string
JSON string representation of the rules configuration.
​
auditTime
string
ISO-8601 timestamp when the change was made.
[
    {
        "key": "social_media",
        "type": "rules",
        "auditInfo": {
            "author": "console",
            "comment": "test",
            "ip": "127.0.0.1"
        },
        "payload": "[{\"interval\":\"2023-01-01T00:00:00.000Z/2023-02-01T00:00:00.000Z\",\"type\":\"dropByInterval\"}]",
        "auditTime": "2023-07-13T18:05:33.066Z"
    },
    {
        "key": "wikipedia_api",
        "type": "rules",
        "auditInfo": {
            "author": "console",
            "comment": "test",
            "ip": "127.0.0.1"
        },
        "payload": "[{\"tieredReplicants\":{\"_default_tier\":2},\"type\":\"loadForever\"}]",
        "auditTime": "2023-07-18T18:10:44.519Z"
    }
]

​
Common rule types

​
Load rules

  • loadForever: Load all segments forever
  • loadByInterval: Load segments within a specific interval
  • loadByPeriod: Load segments within a recent time period

​
Drop rules

  • dropForever: Drop all segments (mark as unused)
  • dropByInterval: Drop segments within a specific interval
  • dropByPeriod: Drop segments older than a time period
  • dropBeforeByPeriod: Drop segments before a time period from now

​
Broadcast rules

  • broadcastForever: Broadcast all segments to all servers
  • broadcastByInterval: Broadcast segments within an interval
  • broadcastByPeriod: Broadcast recent segments by period

Build docs developers (and LLMs) love

Get started for freeTalk to us