Skip to main content

Documentation Index

Fetch the complete documentation index at: https://mintlify.com/holzerjm/civichacks-demo/llms.txt

Use this file to discover all available pages before exploring further.

CivicHacks Demo Hero

​
Build civic AI applications with open source tools

This demo toolkit proves that open source AI is free, powerful, and accessible to anyone with a laptop. Build a complete civic AI application that analyzes real city data—running entirely on your machine with zero API costs. Originally created for CivicHacks 2026 at Boston University, this project demonstrates how to:
  • Run GPT-4-class AI models locally using Ollama
  • Connect AI to real civic datasets using Retrieval Augmented Generation (RAG)
  • Build a polished web application with Gradio
  • Analyze your own data with zero configuration
All processing happens locally on your machine. No data ever leaves your laptop. No API keys required.

​
What you’ll build

In three progressive steps, you’ll create a working civic AI assistant:
1

Local AI inference

Run Llama 3.1 locally and prove that AI works on your laptop for free—no cloud required.
2

RAG with civic data

Connect your AI to real Boston civic datasets covering environment, 311 services, public schools, and criminal justice.
3

Web application

Wrap everything in a shareable web interface with chat, track switching, and cost comparisons.
4

Bring your own data

Drop in any PDF, text file, or dataset and start asking questions immediately.

​
Key features

100% local and free

Everything runs on your machine. Zero API costs—just fractions of a cent in electricity per query.

Real civic datasets

Includes synthetic but realistic Boston data for environment, 311 services, schools, and justice reform.

Production-ready stack

Built with Ollama, LlamaIndex, and Gradio—the same tools used in enterprise AI applications.

Bring your own data

Works with your PDFs, text files, and CSVs out of the box. No code changes needed.

​
The complete stack

This demo uses four open source layers, all running locally: 
┌─────────────────────────────────────────────┐
│              Gradio Web UI                  │  ← Browser-based chat interface
├─────────────────────────────────────────────┤
│          LlamaIndex RAG Pipeline            │  ← Retrieval Augmented Generation
│   ┌──────────────┐    ┌──────────────────┐ │
│   │ Vector Index  │    │ HuggingFace      │ │
│   │ (in-memory)   │    │ Embeddings       │ │
│   └──────────────┘    └──────────────────┘ │
├─────────────────────────────────────────────┤
│            Ollama + Llama 3.1               │  ← Local LLM inference
├─────────────────────────────────────────────┤
│            Civic Data Files                 │  ← .txt datasets per track
└─────────────────────────────────────────────┘
Apple Silicon Macs (M1/M2/M3/M4) are ideal for this demo—unified memory handles Llama 3.1 8B at 15-25 tokens/second. CPU-only machines still work at ~3-5 tokens/second.

​
What the demo covers

The project includes four hackathon tracks with real civic data patterns:
TrackDatasetFocus Area
EcoHackBoston EnvironmentAir quality, heat islands, climate resilience
CityHackBoston 311 ServicesService requests, equity gaps, response times
EduHackBoston Public SchoolsAchievement gaps, absenteeism, tech access
JusticeHackMA Criminal JusticeIncarceration disparities, policing data

​
Quick start

Get the demo running in about 10 minutes:

Quickstart

Get the demo running in 10 minutes

Installation

Detailed setup instructions and requirements

​
Real-world cost comparison

Every query shows you the actual cost: 
⏱️  12.3s · 142 tokens · 11 tok/s
⚡ Local: $0.000009 (0.051 Wh @ 15W) · GPT-4o: $0.0017 (189x more)
The same query that costs fractions of a cent locally would cost nearly 200x more on GPT-4o’s API.

​
What you’ll learn

This demo teaches you how to:
  • Set up and run local LLMs with Ollama
  • Build RAG pipelines with LlamaIndex
  • Create web UIs with Gradio
  • Work with civic datasets
  • Compare local vs. cloud AI costs
  • Deploy AI applications without cloud dependencies
The civic datasets are synthetic but realistic—fabricated for demonstration using real-world patterns. For production apps, use real data from data.boston.gov, mass.gov/open-data, or data.gov.

​
Next steps

Run the quickstart

Get the demo running in 10 minutes

Read installation guide

Detailed setup and requirements

Explore the architecture

Understand how the stack works

View on GitHub

Clone the repository and start building

Build docs developers (and LLMs) love

Get started for freeTalk to us