Skip to main content

Documentation Index

Fetch the complete documentation index at: https://mintlify.com/apursley2012/cartpole/llms.txt

Use this file to discover all available pages before exploring further.

The original Cartpole.ipynb Jupyter notebook is the primary source implementation. It contains the full Python/Keras DQN workflow, training runs, and recorded results that the CartPole // Balance Lab browser demo is built to present. The browser demo is a presentation layer constructed on top of this preserved work — it does not replace it.
The recorded training averages (197.25 and 199.43) belong to the Python/Keras notebook, not the JavaScript recreation. The browser demo is an interactive simulation that follows the same learning structure but runs in a browser-based physics approximation.

​
What the Notebook Contains

Cartpole.ipynb implements the complete DQN training workflow:
  • Python/Keras DQNSolver — Sequential dense network (24 → 24 → 2), compiled with MSE loss and the Adam optimizer
  • CartPole-v1 Gym environment — Standard OpenAI Gym setup providing four state observations and two discrete actions
  • Training loop with experience replay — Episode loop that stores transitions, samples batches, applies the Bellman update, and decays exploration rate
  • ScoreLogger utility — Local helper (scores/score_logger.py) that tracks per-run scores and evaluates progress toward the 195 rolling-average solve criterion
  • Recorded training results — Preserved terminal output from multiple completed runs, including the gamma experiment (avg 195.38) and two baseline solved runs (avg 197.25 and 199.43)

​
Preserved Source Materials

The source/ directory holds the notebook and every original supplied file. Nothing has been overwritten or replaced.
  • Cartpole.ipynb — The main notebook linked from the public site’s “Open Notebook ↗” navigation item
  • README.md — Directory notes
  • original-materials/ — All supplied source files, including duplicate notebook copies (Cartpole-3.ipynb, Cartpole-4.ipynb), the original article draft (cartpole-article-original.md), and the original JavaScript and CSS drafts (cartpole-env-original.js, cartpole-lab-original.js, cartpole-lab-original.css)

​
Relationship to the Browser Demo

The design principle throughout the project is a strict separation between source implementation and presentation layer:
  • Cartpole.ipynb is the source. It contains the actual Keras model, Gym environment, training loop, and recorded results.
  • The website is the presentation layer. It makes the agent’s behavior observable in real time without requiring the visitor to run Python locally.
The JavaScript demo (dqn-agent.js) recreates the same dense-layer structure — 4 inputs, two 24-unit ReLU hidden layers, 2 linear outputs — with the same nine hyperparameter values. Because GitHub Pages cannot execute Python, Gym, or Keras, the neural network forward pass and weight updates are written in vanilla JavaScript. The recorded training results from the notebook are shown on the site as supporting evidence of the original work, not as output of the JavaScript version.

​
How to Access

The notebook is linked from every page of the live site via the “Open Notebook ↗” button in the primary navigation. It can also be accessed directly at source/Cartpole.ipynb within the repository.

​
Source File Structure

cartpole/
cartpole/
├── source/
│   ├── Cartpole.ipynb          # Main notebook
│   ├── README.md
│   └── original-materials/
│       ├── Cartpole-3.ipynb
│       ├── Cartpole-4.ipynb
│       ├── Cartpole.ipynb
│       ├── cartpole-article-original.md
│       ├── cartpole-env-original.js
│       └── cartpole-lab-original.js
├── assets/
│   ├── js/
│   │   ├── cartpole-env.js     # Physics environment
│   │   ├── dqn-agent.js        # Neural network & learning
│   │   └── app.js              # Simulation loop & UI
│   └── css/
│       └── styles.css
├── index.html                  # Live demo
├── case-study.html
├── article.html
└── project-record.html

Development Record

The full notebook source in readable editor panels: imports, configuration, DQNSolver, action selection, experience replay, parameter experiments, and recorded terminal output.

Model Configuration

Network architecture, all nine baseline hyperparameter values, Python dependencies, and a side-by-side comparison of the Python and JavaScript implementations.

Build docs developers (and LLMs) love

Get started for freeTalk to us