Skip to main content

​
Purpose

The TCP Event API provides a high-performance, low-overhead mechanism for monitoring TCP connection state changes and network statistics in real-time using eBPF (extended Berkeley Packet Filter) technology. The library intercepts kernel TCP state transitions and correlates them with process information to deliver comprehensive network telemetry.

​
Library Information

Library Location: /opt/RealTimeKql/lib/libtcpEvent.so Version: 1.03a (tcpTracer Ver 1.03e) Dependencies:
  • BCC (BPF Compiler Collection)
  • pthread library
  • Linux kernel with eBPF support

​
Header Files

To use the TCP Event API in your application, include the main header file: 
#include "event.h"
The header automatically includes:
  • common.h - Data structure definitions (event_t, tcp_event_t, anu_tcp_info)

​
Basic Usage Workflow

​
1. Initialize the BPF Probe

const char *bpf_program = "path/to/bpf_program.c";
AddProbe(bpf_program);
The AddProbe() function spawns a detached thread that:
  • Initializes the BPF subsystem
  • Attaches a kprobe to the tcp_set_state kernel function
  • Opens a perf buffer for event collection
  • Begins polling for TCP state change events

​
2. Wait for Initialization

while (getStatus() == 0) {
    usleep(100000); // Wait 100ms
}
The status flag indicates when the BPF probe is fully initialized and ready to capture events.

​
3. Consume Events

while (1) {
    struct tcp_event_t event = DequeuePerfEvent();
    
    // Process event
    printf("PID: %u, Command: %s\n", event.pid, event.task);
    printf("Connection: %s:%u -> %s:%u\n", 
           event.SADDR, event.SPT, event.DADDR, event.DPT);
    printf("TX: %lu bytes, RX: %lu bytes\n", event.tx_b, event.rx_b);
}
DequeuePerfEvent() blocks until an event is available and returns the next TCP event from the internal queue.

​
4. Cleanup

cleanup();
Detaches the kprobe and cancels background threads.

​
Architecture

​
Event Collection Pipeline

  1. Kernel Probe: BPF probe attached to tcp_set_state() captures TCP state transitions
  2. Perf Buffer: Events flow through a perf ring buffer from kernel to userspace
  3. Event Queue: Userspace handler (handle_output) adds events to a deque (max 1024 events)
  4. Netlink Enrichment: Parallel thread uses netlink socket diagnostics to gather TCP statistics
  5. Consumer API: DequeuePerfEvent() provides blocking access to enriched events

​
Thread Architecture

The library uses three concurrent threads:
ThreadPurposeCreated By
BPF PollerPolls perf buffer for kernel eventsAddProbe() / setupBPF()
Netlink ProbeQueries TCP socket statistics via netlinkDequeuePerfEvent() (lazy init)
Consumer ThreadYour application thread calling DequeuePerfEvent()Your code

​
Thread Safety

The library uses multiple synchronization primitives:
  • pthread_mutex_t mtx: Protects the event deque during enqueue/dequeue operations
  • pthread_cond_t cond: Signals waiting consumers when new events arrive
  • pthread_mutex_t mapMu: Guards the pointer map for event memory management
  • pthread_rwlock_t rwlock: Protects the initialization status flag

​
Thread-Safe Operations

  • DequeuePerfEvent(): Thread-safe, uses mutex and condition variable
  • getStatus(): Thread-safe, uses read-write lock
  • cleanup(): Safe to call from main thread, cancels background threads

​
Important Notes

DequeuePerfEvent() is a blocking call. It will wait indefinitely until an event is available. Design your application to handle this blocking behavior (e.g., dedicate a thread to event collection).
The event queue has a maximum size of 1024 events. If the queue fills, older events are dropped with a “Shedding TCP events” message. Ensure your consumer keeps pace with event generation.

​
Event Flow Control

Queue Overflow Behavior:
  • When queue size exceeds 1024, oldest event is dropped (FIFO)
  • Dropped event memory is properly reclaimed
  • Console warning: "Shedding TCP events.."
Netlink Event Throttling:
  • Netlink thread sleeps for 6.2 seconds (NETLINKNAP) between polling cycles
  • If queue > 1024, netlink events are discarded with warning: "Shedding TCP (Netlink) events.."

​
Supported Protocol Families

  • AF_INET: IPv4 connections (addresses in SADDR/DADDR as dotted-quad strings)
  • AF_INET6: IPv6 connections (addresses in SADDR/DADDR as colon-hex strings)

​
Kernel Compatibility

Requires Linux kernel with:
  • eBPF/BPF support (kernel 4.1+)
  • Kprobe support
  • tcp_set_state symbol (standard in modern kernels)
  • Netlink socket diagnostics (NETLINK_SOCK_DIAG)

​
Performance Characteristics

  • Overhead: Minimal kernel overhead (eBPF in-kernel filtering)
  • Event Latency: Near real-time (microsecond-level timestamps)
  • Memory: ~128KB per 1024 queued events
  • CPU: Polling thread consumes minimal CPU when idle

Build docs developers (and LLMs) love

Get started for freeTalk to us