X’s recommendation algorithm
X’s Recommendation Algorithm is a sophisticated set of services and jobs responsible for serving feeds of posts and content across all X product surfaces. Released as open-source, it represents one of the largest and most complex recommendation systems in production.This algorithm powers billions of recommendations daily across For You Timeline, Search, Explore, and Notifications.
What makes this unique
X’s recommendation system combines multiple state-of-the-art machine learning models, graph-based algorithms, and real-time data processing to deliver personalized content at massive scale:- Real-time processing - User actions are processed in real-time to immediately influence recommendations
- Multi-source candidate generation - Combines in-network and out-of-network content from diverse sources
- Advanced ML models - Uses neural networks, graph embeddings, and community detection algorithms
- Transparency - Open-source codebase allowing public scrutiny and contributions
Explore the system
Architecture
Understand the overall system design and how major components interconnect
How it works
Deep dive into the recommendation pipeline from candidate generation to final ranking
Core services
Explore the key services that power the recommendation system
ML models
Learn about SimClusters, TwHIN, and other machine learning models
Data pipeline
Discover how user signals and actions flow through the system
Development
Get started building and contributing to the project
Key capabilities
Candidate generation
The system narrows down from approximately 1 billion tweets to a few thousand candidates using multiple sources:- Search Index (Earlybird) - Provides ~50% of For You Timeline tweets from in-network content
- User Tweet Entity Graph (UTEG) - Graph-based traversal using GraphJet framework
- Follow Recommendation Service (FRS) - Suggests content from accounts you might want to follow
- Cr-Mixer - Coordinates fetching of out-of-network candidates
Ranking and scoring
Candidates are scored using a multi-stage ranking process:- Light Ranker - Fast initial ranking to pre-filter candidates
- Heavy Ranker - Deep neural network using ~6,000 features
- Feature Hydration - Enriches candidates with signals for ranking
Filtering and mixing
Final timeline construction applies multiple filters and mixing strategies:- Author diversity to prevent timeline dominance
- Content balance between in-network and out-of-network
- Visibility filters for blocked, muted, and NSFW content
- Integration of ads, who-to-follow modules, and prompts
Core technologies
SimClusters
Community detection and sparse embeddings for users and content
TwHIN
Dense knowledge graph embeddings for users and posts
GraphJet
Real-time graph processing for candidate generation
Product Mixer
Framework for building and composing content feeds
Navi
High-performance ML model serving written in Rust
Real Graph
Predicts user-to-user interaction likelihood
User signals
The algorithm uses a rich set of explicit and implicit user signals:Explicit signals
Explicit signals
- Tweet Favorite - Likes indicate strong positive interest
- Retweet - Amplification signal showing endorsement
- Reply - Engagement and conversation participation
- Quote Tweet - Retweet with commentary
- Follow/Unfollow - Account-level preferences
- Bookmark - Save for later interest signal
Implicit signals
Implicit signals
- Tweet Click - Viewing tweet detail pages
- Video Watch - Time spent watching video content
- Profile Visits - Interest in specific accounts
- Notification Opens - Engagement with push notifications
Negative signals
Negative signals
- Tweet Don’t Like - Explicit negative feedback
- Tweet Report - Content quality signals
- Author Mute - Reduce visibility without unfollowing
- Author Block - Strongest negative signal
Product surfaces
The recommendation algorithm currently powers:For You Timeline
Personalized feed combining in-network tweets from people you follow with recommended out-of-network content. Uses the full recommendation pipeline with candidate generation, ranking, filtering, and mixing.Following Timeline
Reverse chronological tweets from accounts you follow, with light filtering and ads integration.Recommended Notifications
Push notifications recommending relevant content, powered by dedicated light and heavy ranking models optimized for notification engagement.Getting started
Understand the architecture
Start with the architecture overview to see how components fit together.
Learn how it works
Read how it works to understand the recommendation pipeline.
Explore the code
Browse the source code at github.com/twitter/the-algorithm
Contributing
X invites the community to submit GitHub issues and pull requests for suggestions on improving the recommendation algorithm. The team is working on tools to manage these suggestions and sync changes to their internal repository.Security concerns should be routed to the official bug bounty program through HackerOne.
Resources
- Engineering blog post - Introduction to the algorithm
- Open source announcement - Why X open-sourced the algorithm
- GitHub repository - Full source code
- SimClusters paper - KDD 2020 publication