Documentation Index
Fetch the complete documentation index at: https://mintlify.com/holzerjm/civichacks-demo/llms.txt
Use this file to discover all available pages before exploring further.
Your starting stack
This is the exact stack from the live demo. It’s a great starting point — you can build a complete hackathon project without adding anything else.Ollama
Run LLMs locally with one command
LlamaIndex
RAG framework — connect AI to your data
Gradio
Build web UIs for ML apps in Python
HuggingFace
Local text embeddings (all-MiniLM-L6-v2)
Local LLMs and model runtimes
Run AI models on your own laptop — no internet, no API keys, no cost.Ollama
Easiest setup, great CLI, wide model support
LM Studio
GUI-based, good for beginners who prefer a visual interface
llamafile
Single-file executables — download one file, run a model
vLLM
High-throughput serving, great for team/server deployments
Jan
Desktop app for local AI, beginner-friendly
Models to try
All available viaollama pull <model>. Experiment with different models to find the best fit for your project.
| Model | Size | Best for | Command |
|---|---|---|---|
| Llama 3.1 8B | 4.7 GB | General purpose (our demo default) | ollama pull llama3.1 |
| Llama 3.2 3B | 2.0 GB | Fast responses, lower RAM | ollama pull llama3.2:3b |
| Mistral 7B | 4.1 GB | Strong general quality, Apache 2.0 license | ollama pull mistral |
| Phi-3 Mini | 2.2 GB | Smallest model that’s still useful | ollama pull phi3:mini |
| DeepSeek-R1 7B | 4.7 GB | Reasoning and analysis tasks | ollama pull deepseek-r1:7b |
| Qwen 2.5 7B | 4.7 GB | Strong multilingual support | ollama pull qwen2.5 |
| Gemma 2 9B | 5.4 GB | Google’s open model, great quality | ollama pull gemma2 |
| CodeLlama 7B | 3.8 GB | Code generation and analysis | ollama pull codellama |
RAG frameworks
RAG (Retrieval Augmented Generation) is the technique we used in the demo — the AI retrieves relevant data before answering, so it cites real facts instead of hallucinating.LlamaIndex
Easiest RAG setup, great docs, what we used
LangChain
Most popular, huge ecosystem, lots of integrations
Haystack
Search-focused RAG, great for document Q&A
DSPy
Programmatic prompting, academic-grade optimization
RAGFlow
Visual RAG pipeline builder with built-in UI
Docling
Document parsing for RAG — converts PDF, DOCX, PPTX, HTML, images to Markdown/JSON
Stick with LlamaIndex for this hackathon — it’s what the demo uses, so you can build on top of the existing code without starting from scratch. Pair it with Docling if you need to parse complex PDFs, PowerPoints, or scanned documents.
Web UI frameworks
Judges want to see a working demo. These frameworks let you build a polished web interface in Python — no JavaScript needed.Gradio
Chat interfaces, ML demos (what we used)
Streamlit
Data dashboards, multi-page apps
Chainlit
Chat-focused apps with auth and history
Panel
Complex data dashboards and visualizations
Mesop
Google’s new Python web UI framework
NiceGUI
Full web apps with more UI control
Use Gradio if you’re building a chat/Q&A interface (fork our Step 3 or Step 5 code). Use Streamlit if you’re building a data exploration dashboard.
Vector databases
The demo uses LlamaIndex’s in-memory vector store, which works great for small-to-medium datasets. If your project needs persistent storage or handles larger data, consider a dedicated vector database.ChromaDB
Simplest to set up, in-memory or persistent, great for hackathons
Faiss
Fastest local search, Facebook/Meta, pure performance
Weaviate
Full-featured, hybrid search, great API
Qdrant
Rust-based, fast, good filtering support
Milvus
Enterprise-grade, handles billions of vectors
Agent frameworks
Want your AI to do more than answer questions? Agent frameworks let AI plan and execute multi-step tasks — like researching a topic, analyzing data, and generating a report.CrewAI
Multi-agent teams with roles and tasks
AutoGen
Microsoft’s multi-agent conversation framework
Agno
Lightweight, fast agent framework
Smolagents
HuggingFace’s simple agent library
Semantic Kernel
Microsoft, works with Python and .NET
LangGraph
LangChain’s graph-based agent orchestration
Free deployment and hosting
Ship your project so judges (and the world) can use it. All of these have free tiers that are more than enough for a hackathon.| Platform | Best for | Free tier | URL |
|---|---|---|---|
| Hugging Face Spaces | Gradio/Streamlit apps | Unlimited public spaces | huggingface.co/spaces |
| Streamlit Cloud | Streamlit apps with GitHub integration | 1 free app | streamlit.io/cloud |
| Render | Web services, APIs, databases | 750 hours/month | render.com |
| Railway | Full-stack deployments | $5 credit/month | railway.app |
| Vercel | Frontends, serverless functions | Generous free tier | vercel.com |
| GitHub Pages | Static sites, documentation | Unlimited | pages.github.com |
| Replit | Full dev environment + deployment | Free tier available | replit.com |
Our demo runs Ollama locally, which won’t work on most cloud platforms. For deployment, you’ll need to swap the local LLM for a cloud API (HuggingFace Inference, Groq, Together AI) or use a platform that supports custom containers.
Useful Python libraries
These are the most common libraries you’ll want topip install during the hackathon, organized by what you’re trying to do.
Data analysis and visualization
| Library | What it does | Install |
|---|---|---|
| pandas | Data manipulation and analysis | pip install pandas |
| matplotlib | Static charts and graphs | pip install matplotlib |
| plotly | Interactive charts and dashboards | pip install plotly |
| seaborn | Statistical visualizations | pip install seaborn |
| altair | Declarative visualization (works great with Streamlit) | pip install altair |
File parsing
| Library | What it does | Install |
|---|---|---|
| pypdf | Read and extract text from PDFs | pip install pypdf |
| python-docx | Read and write Word documents | pip install python-docx |
| openpyxl | Read and write Excel files | pip install openpyxl |
| docling | Advanced document parsing (PDF, DOCX, PPTX, HTML, images) — RAG-optimized with layout understanding and OCR | pip install docling |
| beautifulsoup4 | Parse HTML and scrape web pages | pip install beautifulsoup4 |
APIs and web
| Library | What it does | Install |
|---|---|---|
| requests | Make HTTP requests (API calls, web scraping) | pip install requests |
| fastapi | Build REST APIs quickly | pip install fastapi uvicorn |
| pydantic | Data validation and settings management | pip install pydantic |
| httpx | Modern async HTTP client | pip install httpx |
AI and ML
| Library | What it does | Install |
|---|---|---|
| transformers | HuggingFace model library | pip install transformers |
| sentence-transformers | Embeddings and semantic search | pip install sentence-transformers |
| scikit-learn | Traditional ML algorithms | pip install scikit-learn |
| openai | OpenAI API client (also works with compatible APIs) | pip install openai |
Learning resources
Get up to speed quickly with these resources. Prioritize the ones marked with a star — they’re the most immediately useful for this hackathon.RAG and LLM applications
- LlamaIndex Starter Tutorial ⭐
- Ollama Model Library ⭐
- LangChain Tutorials
- HuggingFace NLP Course
- RAG from Scratch (YouTube) — Search “RAG from scratch LangChain”