JIT optimization

Overview

Just-in-time (JIT) compilation transforms JavaScript from interpreted bytecode into optimized machine code at runtime. Modern JavaScript engines use sophisticated profiling and optimization techniques to identify and accelerate hot code paths, achieving near-native performance.

JIT compilation is optional in engine implementation but critical for production performance. Engines like V8, SpiderMonkey, and JavaScriptCore use multi-tier JIT systems that balance compilation cost with execution speed.

JIT compilation pipeline

Modern engines use a multi-tier approach that progressively optimizes hot code:

┌─────────────────────────────────────────────────────┐
│                Source Code                          │
└──────────────────┬──────────────────────────────────┘
                   v
         ┌─────────────────────┐
         │   Parser + AST      │
         └──────────┬──────────┘
                    v
         ┌─────────────────────┐
         │  Bytecode Generator │
         └──────────┬──────────┘
                    v
    ┌───────────────────────────────────┐
    │      Interpreter (Ignition)       │  <- Fast startup
    │   Execute bytecode + collect      │     No compilation
    │      profiling data               │
    └──────────┬───────────────┬────────┘
               │               │
               │ Hot?          │ Very hot?
               v               v
    ┌──────────────┐  ┌────────────────┐
    │ Baseline JIT │  │ Optimizing JIT │
    │ (Sparkplug)  │  │   (TurboFan)   │
    │              │  │                │
    │ Quick        │  │ Aggressive     │
    │ compilation  │  │ optimization   │
    │ + profiling  │  │ using profile  │
    └──────────────┘  └────────┬───────┘
                               │
                        Assumptions
                        violated?
                               │
                               v
                      ┌─────────────────┐
                      │  Deoptimization │
                      │  Back to        │
                      │  interpreter    │
                      └─────────────────┘

Tier 1: Interpreter
Tier 2: Baseline JIT
Tier 3: Optimizing JIT

Ignition (V8’s interpreter)

Executes bytecode directly
Fast startup (no compilation delay)
Collects type feedback for later optimization
Low memory footprint

// First execution runs in interpreter
function add(a, b) {
  return a + b;
}

add(1, 2); // Interpreter

Sparkplug (V8’s baseline compiler)

Quick compilation to machine code
Minimal optimization (direct bytecode translation)
Continues collecting type feedback
2-5x faster than interpreter

// After a few calls, compiles to baseline
for (let i = 0; i < 100; i++) {
  add(i, i + 1); // Baseline JIT after ~10 iterations
}

TurboFan (V8’s optimizing compiler)

Expensive compilation process
Aggressive optimizations using type feedback
Speculative optimization with guards
10-100x faster than interpreter for numeric code

// After many calls with consistent types
for (let i = 0; i < 10000; i++) {
  add(i, i + 1); // Optimizing JIT after ~1000 iterations
}

// Optimized version assumes integers
// Uses fast integer addition instruction

Compilation has cost. Engines carefully balance when to trigger JIT compilation based on function hotness (call frequency) and compilation cost.

Inline caching

Inline caching (IC) is the most important JIT optimization. It speeds up property access by caching location information from previous lookups.

Property access without IC

Without caching, every property access requires a full lookup:

// Slow: Full prototype chain lookup every time
function getName(obj) {
  return obj.name; // Property lookup algorithm:
                   // 1. Check obj own properties
                   // 2. Check obj.__proto__
                   // 3. Check obj.__proto__.__proto__
                   // ... until found or null
}

Monomorphic inline cache

Monomorphic IC handles one object shape (hidden class):

const user1 = { name: 'Alice', age: 30 };
const user2 = { name: 'Bob', age: 25 };

function getName(obj) {
  return obj.name;
}

// First call: IC miss, perform full lookup
getName(user1); // Cache: { shape: Shape1, offset: 0 }

// Subsequent calls: IC hit (fast path)
getName(user2); // Same shape, use offset 0 directly

// Conceptual inline cache:
// if (obj.shape === Shape1) {
//   return obj[offset: 0]; // Direct memory access!
// } else {
//   fallback_to_full_lookup();
// }

Monomorphic IC can be 10-100x faster than prototype chain traversal by turning property access into a direct memory read at a fixed offset.

Polymorphic inline cache

Polymorphic IC handles 2-4 different shapes:

class User {
  constructor(name) {
    this.name = name;
  }
}

class Admin {
  constructor(name) {
    this.name = name;
    this.role = 'admin';
  }
}

function getName(obj) {
  return obj.name;
}

const user = new User('Alice');
const admin = new Admin('Bob');

// First call: User shape
getName(user); // IC: [{ shape: UserShape, offset: 0 }]

// Second call: Admin shape
getName(admin); // IC: [
                //   { shape: UserShape, offset: 0 },
                //   { shape: AdminShape, offset: 0 }
                // ]

// Conceptual polymorphic IC:
// if (obj.shape === UserShape) {
//   return obj[offset: 0];
// } else if (obj.shape === AdminShape) {
//   return obj[offset: 0];
// } else {
//   fallback_to_full_lookup();
// }

Megamorphic inline cache

When >4 shapes are seen, the IC becomes megamorphic (falls back to generic lookup):

function getName(obj) {
  return obj.name;
}

// Called with many different shapes
for (let i = 0; i < 10; i++) {
  const obj = {};
  obj[`prop${i}`] = i; // Different shape each time!
  obj.name = `name${i}`;
  getName(obj); // IC becomes megamorphic after 4-5 different shapes
}

// Performance cliff: 10x slower than monomorphic

Avoid megamorphic ICs:

Use consistent object shapes (same properties, same order)
Initialize all properties in constructor
Avoid dynamically adding/deleting properties
Use separate functions for different types

IC optimization strategies

Consistent shapes
Property order
Avoid deletion
Class hierarchies

// BAD: Inconsistent shapes
function createUser(name, age) {
  const user = { name };
  if (age) {
    user.age = age; // Different shape!
  }
  return user;
}

// GOOD: Consistent shapes
function createUser(name, age) {
  return {
    name,
    age: age ?? null // Always same properties
  };
}

// BAD: Different property order
const user1 = { name: 'Alice', age: 30 };
const user2 = { age: 25, name: 'Bob' }; // Different shape!

// GOOD: Same property order
const user1 = { name: 'Alice', age: 30 };
const user2 = { name: 'Bob', age: 25 };

// BAD: Deleting properties changes shape
const obj = { a: 1, b: 2, c: 3 };
delete obj.b; // Shape transition!

// GOOD: Set to undefined or use Map
const obj = { a: 1, b: 2, c: 3 };
obj.b = undefined; // Same shape

// Or use Map for dynamic keys
const map = new Map([['a', 1], ['b', 2], ['c', 3]]);
map.delete('b'); // No shape issues

// Initialize all properties early
class User {
  constructor(name, age) {
    this.name = name;
    this.age = age;
    this.email = null; // Reserve property slot
  }
  
  setEmail(email) {
    this.email = email; // No shape change
  }
}

class Admin extends User {
  constructor(name, age) {
    super(name, age);
    this.role = 'admin'; // Predictable shape
  }
}

Type feedback and speculative optimization

Engines collect type information during execution and use it to generate specialized, fast code.

Type feedback collection

function add(a, b) {
  return a + b;
}

// Interpreter tracks types seen:
add(1, 2);        // Feedback: int + int -> int
add(3, 4);        // Feedback: int + int -> int
add(5, 6);        // Feedback: int + int -> int

// After profiling, optimizing JIT generates:
// Specialized integer addition (single CPU instruction)
// Guards check inputs are integers

Speculative optimization

The JIT compiler makes assumptions based on profiled types and inserts guards:

Collect feedback

Interpreter observes types during execution

Speculate

Assume types will remain consistent

Optimize

Generate specialized fast code

Guard

Insert type checks at function entry

Deoptimize

If guards fail, fall back to interpreter

function add(a, b) {
  return a + b;
}

// Optimized version (pseudocode):
function add_optimized(a, b) {
  // Guard: Check assumptions
  if (typeof a !== 'number' || typeof b !== 'number') {
    deoptimize(); // Bailout to interpreter
    return add_unoptimized(a, b);
  }
  
  // Fast path: Integer addition
  return int_add(a, b); // Single CPU instruction
}

Type specialization examples

Numeric operations
Array operations
Object shapes

function calculate(x) {
  return x * 2 + 1;
}

// Integer feedback
for (let i = 0; i < 10000; i++) {
  calculate(i); // JIT sees only integers
}

// Optimized to:
// int_multiply(x, 2) + 1
// (2-3 CPU instructions)

// Deoptimizes if called with:
calculate(3.14);      // Float
calculate("5");       // String
calculate(BigInt(5)); // BigInt

function sum(arr) {
  let total = 0;
  for (let i = 0; i < arr.length; i++) {
    total += arr[i];
  }
  return total;
}

// Feedback: Array of integers
const nums = [1, 2, 3, 4, 5];
sum(nums);

// Optimized version uses:
// - Packed integer array fast path
// - Bounds check elimination (length is constant)
// - Integer addition without type checks

// Deoptimizes if:
sum([1, 2, 3.5]);      // Becomes double array
sum([1, 2, "3"]);      // Becomes generic array
sum([1, 2, , 4]);      // Holey array (sparse)

function getCoordinates(point) {
  return { x: point.x, y: point.y };
}

const p1 = { x: 10, y: 20 };
const p2 = { x: 30, y: 40 };

// Feedback: Monomorphic shape
getCoordinates(p1);
getCoordinates(p2);

// Optimized version:
// - Direct memory offsets (no property lookup)
// - Inline object allocation
// - Eliminates temporary object if unused

// Deoptimizes if called with:
getCoordinates({ y: 1, x: 2 }); // Different shape (property order)
getCoordinates({ x: 1, y: 2, z: 3 }); // Different shape (extra property)

Type-stable code can be 10-100x faster than type-unstable code. Keep function inputs and internal types consistent.

Deoptimization and bailout mechanisms

When assumptions are violated, optimized code must deoptimize back to the interpreter.

What triggers deoptimization

function multiply(x, y) {
  return x * y;
}

// Optimized for integers
for (let i = 0; i < 10000; i++) {
  multiply(i, 2);
}

// Deoptimization triggers:
multiply(3.14, 2);        // 1. Type change (int -> float)
multiply("5", 2);         // 2. Type change (int -> string)
multiply(2, undefined);   // 3. Type change (int -> undefined)

Deoptimization process

Guard failure

Type check or assumption fails in optimized code

State reconstruction

Reconstruct interpreter state from optimized frame

Bailout

Jump back to interpreter at correct position

Continue execution

Interpreter continues with correct state

Remark for optimization

Function may be re-optimized with new feedback

// Deoptimization example
function add(a, b) {
  return a + b; // Line 2
}

// Optimized for integers
add(1, 2);
add(2, 3);

// Deoptimization:
add("hello", "world");

// Engine:
// 1. Detects type mismatch in optimized code
// 2. Reconstructs interpreter state:
//    - PC: line 2
//    - Variables: a="hello", b="world"
// 3. Jumps to interpreter
// 4. Interpreter completes string concatenation
// 5. Function marked for re-optimization with new type info

Deoptimization costs

Deoptimization is expensive:

State reconstruction overhead (100-1000 cycles)
Lost optimization (back to slow interpreter)
Potential re-optimization cost
Repeated deoptimization may prevent future optimization

Soft deopt
Hard deopt
Bailout prevention

// Soft deopt: Rare path, can re-optimize
function process(x) {
  if (typeof x === 'number') {
    return x * 2; // Hot path (optimized)
  }
  return String(x); // Cold path (rare)
}

// 99.9% called with numbers
for (let i = 0; i < 10000; i++) {
  process(i);
}

// Occasional string doesn't break optimization
process("hello"); // Soft deopt, then continues

// Hard deopt: Permanent assumption violation
function calculate(x) {
  return x * x;
}

// Initially called with small integers
for (let i = 0; i < 10000; i++) {
  calculate(i); // Optimized for small ints
}

// Large number changes assumptions
calculate(Number.MAX_SAFE_INTEGER); // Hard deopt

// Future calls may not re-optimize

// Strategy 1: Type guards in source
function safeMath(x) {
  if (typeof x !== 'number') {
    return 0; // Early exit for non-numbers
  }
  return x * x; // Can stay optimized
}

// Strategy 2: Separate code paths
function processNumber(x) {
  return x * 2;
}

function processString(x) {
  return x.repeat(2);
}

function process(x) {
  if (typeof x === 'number') {
    return processNumber(x); // Stays monomorphic
  }
  return processString(x); // Separate optimization
}

Register allocation

Optimizing JIT compilers allocate CPU registers to variables for maximum performance.

Register vs memory access

Performance comparison:

Register access: 1 cycle
L1 cache:        ~4 cycles
L2 cache:        ~12 cycles
L3 cache:        ~40 cycles
RAM:             ~100+ cycles

Goal: Keep hot variables in registers

Register allocation strategies

Linear scan
Graph coloring
Register pressure

// Linear scan allocator (fast compilation)
function calculate(a, b, c) {
  const x = a + b;  // x -> register r1
  const y = b + c;  // y -> register r2
  const z = x * y;  // z -> register r3
  return z;
}

// Assembly-like output:
// r1 = add r_a, r_b
// r2 = add r_b, r_c
// r3 = mul r1, r2
// return r3

Characteristics:

Fast allocation algorithm
Single pass over code
Used in baseline JIT
May spill to stack under register pressure

// Graph coloring (optimizing compiler)
function complex(a, b, c, d) {
  const w = a + b;
  const x = b + c;
  const y = c + d;
  const z = w + x + y;
  return z;
}

// Interference graph:
//   w ----- x
//    \     / \
//     \   /   y
//      \ /   /
//       z   /
//        \ /
//        result

// Coloring (register assignment):
// w -> r1, x -> r2, y -> r1 (w is dead), z -> r3

Characteristics:

Optimal register usage
Considers variable lifetime
Used in optimizing JIT
Expensive to compute

// High register pressure example
function manyVariables() {
  const a = 1, b = 2, c = 3, d = 4;
  const e = 5, f = 6, g = 7, h = 8;
  const i = 9, j = 10, k = 11, l = 12;
  
  // x64 has ~16 general-purpose registers
  // Some reserved for special purposes
  // Realistically ~8-10 available
  
  return a+b+c+d+e+f+g+h+i+j+k+l; // May spill to stack
}

// Optimization: Reduce live ranges
function optimized() {
  let sum = 0;
  sum += 1 + 2;   // r1 = 3
  sum += 3 + 4;   // r1 += 7
  sum += 5 + 6;   // r1 += 11
  // ... only need 1-2 registers
  return sum;
}

Optimizing for register allocation

// BAD: Long variable lifetimes
function processData(data) {
  const temp1 = data.map(x => x * 2);
  const temp2 = data.map(x => x + 1);
  const temp3 = data.map(x => x - 1);
  // All temps live simultaneously - register pressure!
  return temp1.concat(temp2).concat(temp3);
}

// GOOD: Short variable lifetimes
function processData(data) {
  let result = [];
  result = result.concat(data.map(x => x * 2)); // temp1 dies
  result = result.concat(data.map(x => x + 1)); // temp2 dies
  result = result.concat(data.map(x => x - 1)); // temp3 dies
  return result;
}

// BETTER: Eliminate temporaries
function processData(data) {
  return data.flatMap(x => [x * 2, x + 1, x - 1]);
}