Poor performance
Symptoms
Your application’s latency is high and you have already confirmed that the bottleneck is not in dependencies like databases or downstream services. You suspect your application spends significant time running code or processing information. You may also be satisfied with general performance but want to understand which parts of the application can be improved to run faster or more efficiently — for example, to improve user experience or reduce computation cost.What to do
In this scenario, you are interested in code that uses more CPU cycles than others. This document covers two approaches:V8 sampling profiler
Built-in profiler using
--prof. Best for a quick, portable first look at
CPU usage.Linux perf
Low-level CPU profiling with JavaScript, native, and OS-level frames.
Linux only.
V8 sampling profiler
There are many third-party tools available for profiling Node.js applications, but in many cases, the easiest option is to use the Node.js built-in profiler. The built-in profiler uses the profiler inside V8, which samples the stack at regular intervals during program execution. It records the results of these samples, along with important optimization events such as JIT compiles, as a series of ticks:Example application
To illustrate the tick profiler, consider a simple Express application with two handlers — one for adding new users:These handlers are not recommended patterns for authenticating users in
production. They are used purely for illustration. Do not design your own
cryptographic authentication mechanisms — use existing, proven solutions.
Running the profiler
Assume users are complaining about high latency. Run the app with the built-in profiler:ab (ApacheBench):
ab output showing the performance problem:
Processing the tick file
Running with--prof generates a tick file named
isolate-0xnnnnnnnnnnnn-v8.log in the current directory. Process it with
--prof-process:
processed.txt in a text editor. First, look at the summary section:
[C++] section:
PBKDF2, which corresponds to hash generation from user passwords.
To understand the call relationships, examine the [Bottom up (heavy) profile]
section:
parent column percentage tells you the percentage of samples for which
the function in the row above was called by the function in the current row.
Here, _sha1_block_data_order and _malloc_zone_malloc were both called
almost exclusively by pbkdf2. This means password-based hash generation
accounts for all CPU time in the top 3 most sampled functions.
Fixing the bottleneck
The password hash is computed synchronously, blocking the event loop and preventing other incoming requests from being handled. Switch to the asynchronous version ofpbkdf2:
ab run with the asynchronous version yields:
Linux perf
Linux perf provides low-level CPU profiling with JavaScript, native, and OS-level frames. Linux perf is usually available through thelinux-tools-common package.
Through either --perf-basic-prof or --perf-basic-prof-only-functions you
can start a Node.js application that supports perf_events.
--perf-basic-prof always writes to a file (/tmp/perf-PID.map), which can
lead to unbounded disk growth. If that’s a concern, use the
linux-perf module or
--perf-basic-prof-only-functions instead. The latter produces less output
and is a viable option for production profiling.How to use Linux perf
Record events
Record events at the desired frequency. You may want to run a load test
during this step to generate more records for reliable analysis. Close the
perf process with
Ctrl-C when done:Generate a flame graph
The raw output is hard to read. Generate a flame graph for better
visualization. Follow the flame graph guide
from step 6.
Useful links
Memory diagnostics
Node.js (JavaScript) is a garbage-collected language, so memory leaks are possible through retainers. As Node.js applications are usually multi-tenant, business-critical, and long-running, finding and fixing memory issues is essential.My process runs out of memory
Symptoms: Continuously increasing memory usage (which can be fast or slow, over days or even weeks), followed by the process crashing and restarting. The process may run slower than before and restarts may cause some requests to fail (load balancer responds with 502). Side effects:- Process restarts due to memory exhaustion; requests are dropped
- Increased GC activity leads to higher CPU usage and slower response time
- GC blocks the event loop, causing slowness
- Increased memory swapping slows down the process
- May not have enough available memory to get a heap snapshot
My process utilizes memory inefficiently
Symptoms: The application uses an unexpected amount of memory and/or you observe elevated garbage collector activity. Side effects:- An elevated number of page faults
- Higher GC activity and CPU usage
Debugging memory issues
Most memory issues can be solved by determining how much space a specific type of object takes and what variables prevent it from being garbage collected. Knowing the allocation pattern of your program over time also helps.Heap profiler
Capture allocations over time using Allocation Timeline or Sampling Heap
Profiler.
Heap snapshot
Take a snapshot of the heap and inspect it in Chrome DevTools. Compare two
snapshots to find leaks.
GC traces
Use
--trace-gc to observe garbage collection events and identify memory
leaks or excessive GC overhead.Understanding memory
Learn how V8 manages memory and use command-line flags to fine-tune heap
sizes and GC behavior.
Heap profiler
The heap profiler acts on top of V8 to capture allocations over time. Unlike heap snapshots, which capture a point-in-time view, heap profiling lets you understand allocations over a period of time.Allocation timeline
The Allocation Timeline traces every allocation. It has higher overhead than the Sampling Heap Profiler so it is not recommended for use in production.You can use @mmarchini/observe
to start and stop the profiler programmatically.
Open Chrome DevTools Memory tab
Connect to the DevTools instance in Chrome, select the Memory tab, then
select Allocation instrumentation timeline and start profiling.
Sampling heap profiler
The Sampling Heap Profiler tracks the memory allocation pattern and reserved space over time. Because it is sampling-based, its overhead is low enough for use in production systems.You can use the
heap-profiler
module to start and stop the heap profiler programmatically.Open Chrome DevTools Memory tab
Connect to the DevTools instance, then:
- Select the Memory tab.
- Select Allocation sampling.
- Start profiling.
Heap snapshot
You can take a heap snapshot from your running application and load it into Chrome Developer Tools to inspect variables or check retainer size. You can also compare multiple snapshots to see differences over time.Getting a heap snapshot
There are multiple ways to obtain a heap snapshot:- Via inspector
- Via signal flag
- Via writeHeapSnapshot
- Via inspector protocol
Works in all actively maintained versions of Node.js.Run node with
--inspect and open the inspector. Go to the Memory tab
and take a heap snapshot.Finding a memory leak with heap snapshots
Compare two snapshots to find a memory leak. Follow these steps to produce a clean diff:Let the process bootstrap
Let the process load all sources and finish bootstrapping. This should take
a few seconds at most.
Exercise the suspect functionality
Start using the functionality you suspect is leaking memory. It will likely
make some initial allocations that are not the leaking ones.
Continue using the functionality
Continue using the functionality for a while, preferably without running
anything else in between.
Take the second snapshot
Take another heap snapshot. The difference between the two should mostly
contain what is leaking.
Compare in Chrome DevTools
Open Chromium/Chrome DevTools and go to the Memory tab. Load the older
snapshot file first, then the newer one second. Select the newer snapshot
and switch the dropdown at the top from Summary to Comparison. Look
for large positive deltas and explore the references in the bottom panel.
Tracing garbage collection
This section covers the fundamentals of garbage collection traces. By the end, you will be able to:- Enable GC traces in your Node.js application
- Interpret traces
- Identify potential memory issues
When GC is running, your code is not. Knowing how often and how long garbage
collection runs — and what the outcome is — helps you spot performance
problems caused by excessive GC pressure.
Setup
For the examples in this section, use the following script:Running with GC traces
Use the--trace-gc flag to print GC events to the console:
Reading a trace line
Each--trace-gc line follows this structure:
| Token | Interpretation |
|---|---|
13973 | PID of the running process |
0x110008000 | Isolate (JS heap instance) |
44 ms | Time since the process started, in ms |
Scavenge | Type/phase of GC |
2.4 | Heap used before GC, in MB |
(3.2) | Total heap before GC, in MB |
2.0 | Heap used after GC, in MB |
(4.2) | Total heap after GC, in MB |
0.5 / 0.0 ms (average mu = 1.000, current mu = 1.000) | Time spent in GC, in ms |
allocation failure | Reason for GC |
GC event types
Scavenge collects objects in the “new” space (short-lived objects). The new space is designed to be small and fast for garbage collection. Objects not collected after two Scavenge operations are promoted to the old space. Mark-sweep collects objects from the “old” space (long-lived objects). It operates in two phases:- Mark: marks living objects as black and dead objects as white.
- Sweep: scans for white objects and converts them to free space.
Detecting a memory leak
If you see manyMark-sweep events where the amount of memory collected after
each event is insignificant, you likely have a memory leak.
To get context on bad allocations:
Run until OOM
Run the program until it hits an out-of-memory error. The log will show the
failing context:
Detecting slowness from GC
Use these heuristics when reviewing--trace-gc output:
- If the time between two GC events is less than the time spent in GC, the application is severely starving.
- If both the time between GC events and the time spent in GC are very high, the application can probably use a smaller heap.
- If the time between GC events is much greater than the time spent in GC, the application is relatively healthy.
Fixing the leak
Instead of accumulating entries in aSet in memory, write them to a file:
Mark-sweepevents appear less frequently.- Memory footprint stays below 25 MB versus 130+ MB with the first script.
Tracing GC programmatically
- v8 module
- Performance hooks
Use the
v8 module to enable or disable GC tracing at runtime without
restarting the process:Understanding and tuning memory
Node.js, built on Google’s V8 JavaScript engine, offers a powerful runtime for server-side JavaScript. As your applications grow, managing memory becomes critical for maintaining optimal performance and avoiding problems like memory leaks or crashes.How V8 manages memory
V8 divides memory into several parts, with two primary areas being the heap and the stack.The heap
V8’s memory management is based on the generational hypothesis: most objects die young. Therefore, it separates the heap into generations:- New space — where new, short-lived objects are allocated. Garbage collection occurs frequently to reclaim memory quickly. For example, a high-throughput API generating a temporary object per request will have these objects cleaned up via frequent minor GC cycles.
- Old space — where objects that survive multiple GC cycles in the new space are promoted. These are usually long-lived objects such as user sessions, cache data, or persistent state. GC in this space occurs less often but is more resource-intensive. As the number of concurrent users grows, the old space can fill up and cause out-of-memory errors or slower response times.
The stack
The stack stores local variables and function call information. It operates on a Last In, First Out (LIFO) principle. Each function call pushes a new frame; returning pops it. The stack is smaller and faster than the heap but has a limited size — excessive recursion can cause a stack overflow.Monitoring memory usage
Theprocess.memoryUsage() method shows how much memory your Node.js process
is using:
| Field | Description |
|---|---|
rss | Resident Set Size: total memory allocated to the process, including heap and other areas |
heapTotal | Total memory allocated for the heap |
heapUsed | Memory currently in use within the heap |
external | Memory used by external resources like C++ library bindings |
arrayBuffers | Memory allocated to various Buffer-like objects |
heapUsed steadily grows without being released, it could indicate a memory
leak.
Command-line flags for memory tuning
--max-old-space-size
Sets the limit on the old space size in megabytes. Useful when your application
holds a large amount of persistent data:
--max-semi-space-size
Controls the size of the new space. Increasing this reduces the frequency of
minor GC cycles, which helps in high-throughput environments with frequent
short-lived object creation:
--gc-interval
Adjusts how frequently GC cycles occur. Use with caution: too low a value
causes performance degradation from excessive GC:
--expose-gc
Exposes a global.gc() function that lets you manually trigger garbage
collection — for example, after processing a large batch of data: