Environment variables and CLI flags
| Environment Variable | CLI Flag | Description |
|---|---|---|
BATCH_IDLE_DURATION | --batch-idle-duration | The maximum amount of time with no new pending pods that if exceeded ends the current batching window. If pods arrive faster than this time, the batching window will be extended up to the maxDuration. If they arrive slower, the pods will be batched separately. (default = 1s) |
BATCH_MAX_DURATION | --batch-max-duration | The maximum length of a batch window. The longer this is, the more pods we can consider for provisioning at one time which usually results in fewer but larger nodes. (default = 10s) |
CLUSTER_CA_BUNDLE | --cluster-ca-bundle | Cluster CA bundle for nodes to use for TLS connections with the API server. If not set, this is taken from the controller’s TLS configuration. |
CLUSTER_ENDPOINT | --cluster-endpoint | The external kubernetes cluster endpoint for new nodes to connect with. If not specified, will discover the cluster endpoint using DescribeCluster API. |
CLUSTER_NAME | --cluster-name | [REQUIRED] The kubernetes cluster name for resource discovery. |
CPU_REQUESTS | --cpu-requests | CPU requests in millicores on the container running the controller. (default = 1000) |
DISABLE_CLUSTER_STATE_OBSERVABILITY | --disable-cluster-state-observability | Disable cluster state metrics and events. |
DISABLE_CONTROLLER_WARMUP | --disable-controller-warmup | Disable controller warmup which starts controller sources before leader election is won. Controller warmup pre-populates caches and improves leader failover time. |
DISABLE_DRY_RUN | --disable-dry-run | If true, then disable dry run validation for EC2NodeClasses. |
DISABLE_LEADER_ELECTION | --disable-leader-election | Disable the leader election client before executing the main loop. Disable when running replicated components for high availability is not desired. |
EKS_CONTROL_PLANE | --eks-control-plane | Marking this true means that your cluster is running with an EKS control plane and Karpenter should attempt to discover cluster details from the DescribeCluster API. |
ENABLE_PROFILING | --enable-profiling | Enable the profiling on the metric endpoint. |
FEATURE_GATES | --feature-gates | Optional features can be enabled / disabled using feature gates. Current options are: NodeRepair, ReservedCapacity, SpotToSpotConsolidation, NodeOverlay, and StaticCapacity. (default = NodeRepair=false,ReservedCapacity=true,SpotToSpotConsolidation=false,NodeOverlay=false,StaticCapacity=false) |
HEALTH_PROBE_PORT | --health-probe-port | The port the health probe endpoint binds to for reporting controller health. (default = 8081) |
IGNORE_DRA_REQUESTS | --ignore-dra-requests | When set, Karpenter will ignore pods’ DRA requests during scheduling simulations. Note: This flag will be removed once formal DRA support is GA in Karpenter. |
INTERRUPTION_QUEUE | --interruption-queue | Interruption queue is the name of the SQS queue used for processing interruption events from EC2. Interruption handling is disabled if not specified. Enabling interruption handling may require additional permissions on the controller service account. |
ISOLATED_VPC | --isolated-vpc | If true, then assume we can’t reach AWS services which don’t have a VPC endpoint. This also has the effect of disabling look-ups to the AWS on-demand pricing endpoint. |
KARPENTER_SERVICE | --karpenter-service | The Karpenter Service name for the dynamic webhook certificate. |
KUBE_CLIENT_BURST | --kube-client-burst | The maximum allowed burst of queries to the kube-apiserver. (default = 300) |
KUBE_CLIENT_QPS | --kube-client-qps | The smoothed rate of qps to kube-apiserver. (default = 200) |
LEADER_ELECTION_NAME | --leader-election-name | Leader election name to create and monitor the lease if running outside the cluster. (default = karpenter-leader-election) |
LEADER_ELECTION_NAMESPACE | --leader-election-namespace | Leader election namespace to create and monitor the lease if running outside the cluster. |
LOG_ERROR_OUTPUT_PATHS | --log-error-output-paths | Optional comma separated paths for logging error output. (default = stderr) |
LOG_LEVEL | --log-level | Log verbosity level. Can be one of debug, info, or error. (default = info) |
LOG_OUTPUT_PATHS | --log-output-paths | Optional comma separated paths for directing log output. (default = stdout) |
MEMORY_LIMIT | --memory-limit | Memory limit on the container running the controller. The GC soft memory limit is set to 90% of this value. (default = -1) |
METRICS_PORT | --metrics-port | The port the metric endpoint binds to for operating metrics about the controller itself. (default = 8080) |
MIN_VALUES_POLICY | --min-values-policy | Min values policy for scheduling. Options include Strict for existing behavior where min values are strictly enforced or BestEffort where Karpenter relaxes min values when it isn’t satisfied. (default = Strict) |
PREFERENCE_POLICY | --preference-policy | How the Karpenter scheduler should treat preferences. Can be one of Ignore and Respect. (default = Respect) |
RESERVED_ENIS | --reserved-enis | Reserved ENIs are not included in the calculations for max-pods or kube-reserved. Most often used in VPC CNI custom networking setup. (default = 0) |
VM_MEMORY_OVERHEAD_PERCENT | --vm-memory-overhead-percent | The VM memory overhead as a percent that will be subtracted from the total memory for all instance types when cached information is unavailable. (default = 0.075) |
Feature gates
Karpenter uses feature gates to enable and disable optional functionality. You can configure feature gates via the--feature-gates CLI flag or the FEATURE_GATES environment variable.
| Feature | Default | Stage | Since | Until |
|---|---|---|---|---|
Drift | false | Alpha | v0.21.x | v0.32.x |
Drift | true | Beta | v0.33.x | v0.37.x |
SpotToSpotConsolidation | false | Alpha | v0.34.x | — |
NodeRepair | false | Alpha | v1.1.x | — |
ReservedCapacity | false | Alpha | v1.3.x | v1.5.x |
ReservedCapacity | true | Beta | v1.6.x | — |
NodeOverlay | false | Alpha | v1.7.x | — |
StaticCapacity | false | Alpha | v1.8.x | — |
In v1, drift has been promoted to stable and the feature gate removed. You can continue to control drift behavior using disruption budgets with the
Drifted reason:Configuring settings via Helm
When deploying Karpenter with Helm, you can set environment variables using.Values.controller.env and pass CLI flags using .Values.controller.extraArgs.
helm install or helm upgrade:
Batching parameters
The batching parameters control how Karpenter batches an incoming stream of pending pods. Reducing these values may trade off a slightly faster time from pending pod to node launch, in exchange for launching smaller nodes. Increasing the values can do the inverse. For a standard deployment scale-up, the pods arrive at the QPS setting of thekube-controller-manager, and the default values are typically fine. These settings are intended for use cases where other systems may create large numbers of pods over a period of many seconds or minutes and there is a desire to batch them together.
Batch idle duration
Batch idle duration
The batch idle duration is the period of time that a new pending pod extends the current batching window. This can be increased to handle scenarios where pods arrive slower than one second apart, but it would be preferable if they were batched together onto a single larger node.This value is expressed as a string like
CLI flag:
Default:
10s, 1m, or 2h45m. Valid time units are ns, us (or µs), ms, s, m, h.Environment variable: BATCH_IDLE_DURATIONCLI flag:
--batch-idle-durationDefault:
1sBatch max duration
Batch max duration
The batch max duration is the maximum period of time a batching window can be extended to. Increasing this value will allow the maximum batch window size to increase to collect more pending pods into a single batch at the expense of a longer delay from when the first pending pod was created.This value is expressed as a string like
CLI flag:
Default:
10s, 1m, or 2h45m. Valid time units are ns, us (or µs), ms, s, m, h.Environment variable: BATCH_MAX_DURATIONCLI flag:
--batch-max-durationDefault:
10s