Skip to main content

Documentation Index

Fetch the complete documentation index at: https://mintlify.com/cloudflare/pingora/llms.txt

Use this file to discover all available pages before exploring further.

In this guide you will build a working HTTP load balancer that distributes incoming requests across two upstream backends — 1.1.1.1:443 and 1.0.0.1:443 — in a round-robin fashion. Along the way you will add TCP health checks so that a broken backend is automatically removed from rotation, wire up CLI argument parsing, write a configuration file, and perform a zero-downtime binary upgrade. By the end you will have a solid foundation you can adapt for your own upstream services.
1

Create a new Cargo project

cargo new load_balancer
cd load_balancer
Open Cargo.toml and add the two required dependencies. The lb feature flag pulls in pingora-load-balancing and pingora-proxy; openssl activates the OpenSSL TLS backend.
[dependencies]
async-trait = "0.1"
pingora = { version = "0.8.0", features = ["openssl", "lb"] }
2

Create a Pingora Server

A Pingora Server is a process that can host one or many services. It handles configuration and CLI argument parsing, daemonisation, OS signal handling, and graceful restart or shutdown. The idiomatic pattern is to construct it in main() and call run_forever() at the end, which spawns the Tokio runtime threads and blocks until the server is ready to exit.Replace the contents of src/main.rs with:
use async_trait::async_trait;
use pingora::prelude::*;
use std::sync::Arc;

fn main() {
    let mut my_server = Server::new(None).unwrap();
    my_server.bootstrap();
    my_server.run_forever();
}
This compiles and runs cleanly, but does nothing useful yet — there are no services attached.
3

Implement ProxyHttp for your load balancer

Define the LB struct that wraps a LoadBalancer<RoundRobin> and implement the ProxyHttp trait for it. The only required method is upstream_peer(), which returns the address to proxy each request to. The LoadBalancer::select() call advances the round-robin cursor and returns the next healthy backend.
pub struct LB(Arc<LoadBalancer<RoundRobin>>);

#[async_trait]
impl ProxyHttp for LB {
    /// For this small example, we don't need context storage.
    type CTX = ();
    fn new_ctx(&self) -> () {
        ()
    }

    async fn upstream_peer(
        &self,
        _session: &mut Session,
        _ctx: &mut (),
    ) -> Result<Box<HttpPeer>> {
        let upstream = self
            .0
            .select(b"", 256) // hash doesn't matter for round robin
            .unwrap();

        println!("upstream peer is: {upstream:?}");

        // HttpPeer::new(address, tls, sni)
        // `true`  — enable TLS, required for the 1.1.1.1 HTTPS backends.
        // The SNI `one.one.one.one` must match the upstream certificate.
        let peer = Box::new(HttpPeer::new(upstream, true, "one.one.one.one".to_string()));
        Ok(peer)
    }
}
4

Add the upstream_request_filter hook

The 1.1.1.1 demo backends require a Host header to be present. The upstream_request_filter() callback fires after the upstream connection is established but before the request headers are sent — the right place to inject or rewrite headers.Add this method inside the same impl ProxyHttp for LB block:
async fn upstream_request_filter(
    &self,
    _session: &mut Session,
    upstream_request: &mut RequestHeader,
    _ctx: &mut Self::CTX,
) -> Result<()> {
    upstream_request
        .insert_header("Host", "one.one.one.one")
        .unwrap();
    Ok(())
}
5

Create the proxy service and wire it to the server

http_proxy_service() wraps your LB in a Service that speaks HTTP proxy protocol. add_tcp() tells it which address and port to listen on. Finally, add_service() registers the service with the Server so it is started when run_forever() is called.Replace your main() function with the complete version:
fn main() {
    let mut my_server = Server::new(None).unwrap();
    my_server.bootstrap();

    let upstreams =
        LoadBalancer::try_from_iter(["1.1.1.1:443", "1.0.0.1:443"]).unwrap();

    let mut lb = http_proxy_service(&my_server.configuration, LB(Arc::new(upstreams)));
    lb.add_tcp("0.0.0.0:6188");

    my_server.add_service(lb);
    my_server.run_forever();
}

​
Test It

Start the load balancer: 
cargo run
In a second terminal, send a few requests and watch the server logs alternate between the two backends: 
curl 127.0.0.1:6188 -svo /dev/null
You should see output similar to the following in the server terminal, confirming that traffic is being distributed across both upstreams: 
upstream peer is: Backend { addr: Inet(1.0.0.1:443), weight: 1 }
upstream peer is: Backend { addr: Inet(1.1.1.1:443), weight: 1 }
upstream peer is: Backend { addr: Inet(1.0.0.1:443), weight: 1 }
upstream peer is: Backend { addr: Inet(1.1.1.1:443), weight: 1 }
You can also open http://localhost:6188 in your browser to verify.

​
Adapting to Your Own Upstreams

The example above is deliberately tuned for the 1.1.1.1 demo backends. Pointing the load balancer at your own services usually requires two changes.
When proxying to local plain-HTTP servers (e.g. 127.0.0.1:9000), you must disable TLS in the HttpPeer constructor. Leaving tls = true against a non-TLS upstream will cause every connection attempt to fail with a 502 Bad Gateway.
1. Change the backend addresses in main(): 
let upstreams =
    LoadBalancer::try_from_iter(["127.0.0.1:9000", "127.0.0.1:9001"]).unwrap();
2. Disable TLS in upstream_peer(). Pass false for the tls argument and an empty string for the SNI — it is ignored for plain-HTTP connections: 
let peer = Box::new(HttpPeer::new(upstream, false, String::new()));
Never pass tls = true when your upstream speaks plain HTTP. The TLS handshake will fail and every request will return a 502. Conversely, if your upstream requires TLS (e.g. an external HTTPS API), keep tls = true and provide the correct SNI hostname.
3. Update or remove the Host header override in upstream_request_filter(). The one.one.one.one value is specific to the demo backends; set it to whatever hostname your upstream expects, or delete the method entirely to let Pingora forward the original Host header from the client.

​
Add Health Checks

Without health checks, a dead backend is still selected by the round-robin scheduler — every third request (in a three-peer pool) will fail with a 502. Pingora’s TcpHealthCheck probes each backend in the background and removes unhealthy peers from rotation automatically. First, add a third backend that is guaranteed to be broken so you can observe the effect: 
let mut upstreams =
    LoadBalancer::try_from_iter(["1.1.1.1:443", "1.0.0.1:443", "127.0.0.1:343"]).unwrap();
Then attach the health-check service: 
use pingora::prelude::*;
use std::time::Duration;

fn main() {
    let mut my_server = Server::new(None).unwrap();
    my_server.bootstrap();

    let mut upstreams =
        LoadBalancer::try_from_iter(["1.1.1.1:443", "1.0.0.1:443", "127.0.0.1:343"]).unwrap();

    // Attach a TCP health check that probes each backend every second.
    let hc = TcpHealthCheck::new();
    upstreams.set_health_check(hc);
    upstreams.health_check_frequency = Some(Duration::from_secs(1));

    // Wrap the load balancer in a background service so the health checks run
    // on their own Tokio task. `background.task()` gives back the Arc handle
    // used by the proxy service.
    let background = background_service("health check", upstreams);
    let upstreams = background.task();

    let mut lb = http_proxy_service(&my_server.configuration, LB(upstreams));
    lb.add_tcp("0.0.0.0:6188");

    // Register both the proxy service and the health-check background service.
    my_server.add_service(lb);
    my_server.add_service(background);

    my_server.run_forever();
}
With this in place, 127.0.0.1:343 will be marked unhealthy within one second of startup and will never receive traffic. If it later becomes reachable it will be re-added to rotation automatically within the next health-check cycle.

​
CLI Options and Daemon Mode

Passing Some(Opt::parse_args()) to Server::new() enables Pingora’s built-in CLI argument parsing: 
fn main() {
    let opt = Opt::parse_args();
    let mut my_server = Server::new(Some(opt)).unwrap();
    // ...
}
Run with -h to see all available flags: 
cargo run -- -h
To run the process in the background, pass the -d / --daemon flag: 
cargo run -- -d
Send SIGTERM for a graceful shutdown (the server stops accepting new connections but finishes all in-flight requests before exiting): 
pkill -SIGTERM load_balancer
Send SIGQUIT to initiate a graceful upgrade (the server hands its listening sockets to a new process — see Graceful Upgrade below): 
pkill -SIGQUIT load_balancer

​
Configuration File

Pingora can read runtime settings from a YAML configuration file. Create conf.yaml in your project directory: 
---
version: 1
threads: 2
pid_file: /tmp/load_balancer.pid
error_log: /tmp/load_balancer_err.log
upgrade_sock: /tmp/load_balancer.sock
KeyDescription
versionConfig schema version — always 1 for now.
threadsNumber of Tokio worker threads.
pid_filePath where the daemon writes its PID.
error_logPath for the error log file.
upgrade_sockUnix socket used to hand over listening sockets during a graceful upgrade.
Start the daemon with the config file and enable INFO-level logging so that log entries are written to error_log: 
RUST_LOG=INFO cargo run -- -c conf.yaml -d
Check the PID of the running daemon: 
cat /tmp/load_balancer.pid

​
Graceful Upgrade

Pingora supports zero-downtime binary upgrades on Linux. The old process listens for SIGQUIT, then hands its open listening sockets to the new process over the upgrade_sock Unix socket. The old process continues handling its in-flight requests until they are all complete before exiting. From a client’s perspective the listening socket is never closed. To upgrade a running daemon to a newly compiled binary: 
# 1. Signal the running server to enter upgrade-wait mode.
pkill -SIGQUIT load_balancer

# 2. Start the new binary in upgrade mode (reads sockets from the old process).
RUST_LOG=INFO cargo run -- -c conf.yaml -d --upgrade
The -u / --upgrade flag tells the new server to connect to upgrade_sock and receive the listening sockets from the old server rather than binding them fresh.

Build docs developers (and LLMs) love

Get started for freeTalk to us