Skip to main content

Documentation Index

Fetch the complete documentation index at: https://mintlify.com/apursley2012/cartpole/llms.txt

Use this file to discover all available pages before exploring further.

CartPoleEnvironment implements the CartPole-v1 physics in vanilla JavaScript. It models a cart on a frictionless track with a rigid pole attached at a pivot point, computing real inverted-pendulum dynamics on every time step. The environment is self-contained — no external libraries are required — and its interface mirrors the OpenAI Gym CartPole-v1 convention of reset() and step().

​
Class Overview

The complete source of cartpole-env.js: 
class CartPoleEnvironment {
  constructor() {
    this.gravity = 9.8;
    this.massCart = 1.0;
    this.massPole = 0.1;
    this.totalMass = this.massPole + this.massCart;
    this.length = 0.5;
    this.poleMassLength = this.massPole * this.length;
    this.forceMag = 10.0;
    this.tau = 0.02;
    this.thetaThresholdRadians = 12 * Math.PI / 180;
    this.xThreshold = 2.4;
    this.maxSteps = 500;
    this.reset();
  }

  reset() {
    this.state = Array.from({ length: 4 }, () => Math.random() * 0.1 - 0.05);
    this.steps = 0;
    return [...this.state];
  }

  step(action) {
    const [x, xDot, theta, thetaDot] = this.state;
    const force = action === 1 ? this.forceMag : -this.forceMag;
    const cosTheta = Math.cos(theta);
    const sinTheta = Math.sin(theta);
    const temp = (force + this.poleMassLength * thetaDot * thetaDot * sinTheta) / this.totalMass;
    const thetaAcc = (this.gravity * sinTheta - cosTheta * temp) /
      (this.length * (4 / 3 - this.massPole * cosTheta * cosTheta / this.totalMass));
    const xAcc = temp - this.poleMassLength * thetaAcc * cosTheta / this.totalMass;

    this.state = [
      x + this.tau * xDot,
      xDot + this.tau * xAcc,
      theta + this.tau * thetaDot,
      thetaDot + this.tau * thetaAcc
    ];
    this.steps += 1;

    const terminal = Math.abs(this.state[0]) > this.xThreshold ||
      Math.abs(this.state[2]) > this.thetaThresholdRadians ||
      this.steps >= this.maxSteps;

    return {
      state: [...this.state],
      reward: terminal && this.steps < this.maxSteps ? -1 : 1,
      terminal,
      score: this.steps
    };
  }
}

​
Constructor Properties

Every property is set in the constructor and remains constant throughout an episode. reset() is called automatically at the end of the constructor to initialize this.state and this.steps.
​
gravity
number
default:"9.8"
Gravitational acceleration (m/s²). Used directly in the pole angular acceleration formula.
​
massCart
number
default:"1.0"
Mass of the cart (kg).
​
massPole
number
default:"0.1"
Mass of the pole (kg).
​
totalMass
number
default:"1.1"
Combined mass of cart and pole: massPole + massCart = 0.1 + 1.0 = 1.1 kg. Computed in the constructor rather than hard-coded.
​
length
number
default:"0.5"
Half the length of the pole (m). The physics equations use this as the distance from the pivot to the pole’s center of mass.
​
poleMassLength
number
default:"0.05"
Pre-computed convenience product: massPole * length = 0.1 × 0.5 = 0.05. Appears repeatedly in the acceleration equations.
​
forceMag
number
default:"10.0"
Magnitude of the force applied to the cart on each step (N). A left action applies -forceMag; a right action applies +forceMag.
​
tau
number
default:"0.02"
Seconds per simulation step. Used as the Euler integration time-delta to update position and velocity from acceleration.
​
thetaThresholdRadians
number
default:"≈ 0.2094 rad (12°)"
Maximum allowable pole angle before the episode terminates. Computed as 12 * Math.PI / 180.
​
xThreshold
number
default:"2.4"
Maximum allowable cart position (m) in either direction from center. If |x| > 2.4 the episode ends.
​
maxSteps
number
default:"500"
Maximum number of steps per episode. Reaching this count is considered a successful episode and does not yield a negative reward.

​
reset()

Resets the environment to a fresh starting state. Called automatically by the constructor and by app.js at the start of every new episode. 
reset() {
  this.state = Array.from({ length: 4 }, () => Math.random() * 0.1 - 0.05);
  this.steps = 0;
  return [...this.state];
}
Each of the four state values — [x, xDot, theta, thetaDot] — is drawn independently from a uniform distribution over [-0.05, 0.05]. Returns number[] — a shallow copy of the initial 4-element state array.

​
step(action)

Advances the simulation by one time step using Euler integration of the inverted-pendulum equations of motion. 
step(action) {
  const [x, xDot, theta, thetaDot] = this.state;
  const force = action === 1 ? this.forceMag : -this.forceMag;
  const cosTheta = Math.cos(theta);
  const sinTheta = Math.sin(theta);
  const temp = (force + this.poleMassLength * thetaDot * thetaDot * sinTheta) / this.totalMass;
  const thetaAcc = (this.gravity * sinTheta - cosTheta * temp) /
    (this.length * (4 / 3 - this.massPole * cosTheta * cosTheta / this.totalMass));
  const xAcc = temp - this.poleMassLength * thetaAcc * cosTheta / this.totalMass;

  this.state = [
    x + this.tau * xDot,
    xDot + this.tau * xAcc,
    theta + this.tau * thetaDot,
    thetaDot + this.tau * thetaAcc
  ];
  this.steps += 1;

  const terminal = Math.abs(this.state[0]) > this.xThreshold ||
    Math.abs(this.state[2]) > this.thetaThresholdRadians ||
    this.steps >= this.maxSteps;

  return {
    state: [...this.state],
    reward: terminal && this.steps < this.maxSteps ? -1 : 1,
    terminal,
    score: this.steps
  };
}

​
Input

​
action
number
required
Integer action to apply this step.
ValueDirectionForce applied
0Left-10.0 N
1Right+10.0 N

​
Physics Update Equations

The step computes two accelerations from the current state and the applied force, then integrates them forward by tau = 0.02 seconds: Intermediate quantity: 
temp = (force + poleMassLength × thetaDot² × sin(theta)) / totalMass
Angular acceleration of pole: 
thetaAcc = (gravity × sin(theta) − cos(theta) × temp)
           / (length × (4/3 − massPole × cos²(theta) / totalMass))
Linear acceleration of cart: 
xAcc = temp − poleMassLength × thetaAcc × cos(theta) / totalMass
Euler integration (τ = 0.02 s): 
x′       = x     + tau × xDot
xDot′    = xDot  + tau × xAcc
theta′   = theta + tau × thetaDot
thetaDot′= thetaDot + tau × thetaAcc

​
Return Value

step() returns a plain object with four fields: 
{
  state: number[],   // [x, xDot, theta, thetaDot] after integration
  reward: number,    // 1 normally; -1 on a failure terminal
  terminal: boolean, // true if the episode has ended
  score: number      // this.steps (total steps taken this episode)
}

​
Terminal Conditions

The episode ends (terminal = true) when any of the following is true:
ConditionExpression
Cart out of boundsMath.abs(state[0]) > 2.4
Pole too far from verticalMath.abs(state[2]) > thetaThresholdRadians (≈ 0.2094 rad / 12°)
Max steps reachedthis.steps >= 500

​
Reward Schedule

reward: terminal && this.steps < this.maxSteps ? -1 : 1
OutcomeReward
Survived this step (non-terminal)+1
Episode ended by reaching 500 steps+1
Episode ended by going out of bounds or tilting too far-1
The physics equations closely mirror the OpenAI Gym CartPole-v1 implementation, adapted from Barto, Sutton & Anderson (1983). The state vector, threshold values, force magnitude, and time step are all identical to the Gym reference environment.

Build docs developers (and LLMs) love

Get started for freeTalk to us