Home Mixer

Overview

Home Mixer is the primary orchestration service that powers the For You feed on X. It coordinates multiple subsystems to retrieve, rank, and filter posts, returning a personalized feed for each user. The service leverages the Candidate Pipeline framework to compose a multi-stage recommendation pipeline, combining both in-network content (from Thunder) and out-of-network content (from Phoenix Retrieval).

Architecture

Home Mixer exposes a gRPC endpoint (ScoredPostsService) that processes feed requests through a series of pipeline stages:

pub struct HomeMixerServer {
    phx_candidate_pipeline: Arc<PhoenixCandidatePipeline>,
}

Pipeline Stages

The PhoenixCandidatePipeline implements the complete feed generation flow:

Query Hydration

Fetch user context before retrieving candidates:

User Action Sequence: Recent engagement history (likes, replies, reposts)
User Features: Following list, preferences, language, country

Candidate Sourcing

Retrieve candidates from multiple sources in parallel:

Thunder: In-network posts from followed accounts
Phoenix Retrieval: Out-of-network posts discovered via ML-based similarity search

Hydration

Enrich candidates with additional metadata:

Core post data (text, media, timestamps)
Author information (username, verification status)
Video duration for video posts
Subscription eligibility

Pre-Scoring Filters

Remove ineligible candidates:

Duplicates and reposts of same content
Posts older than retention threshold
Self-posts (user’s own tweets)
Blocked/muted authors
Muted keywords
Previously seen/served posts

Scoring

Apply multiple scorers sequentially:

Phoenix Scorer: ML predictions from transformer model
Weighted Scorer: Combine predictions into relevance score
Author Diversity Scorer: Attenuate repeated authors
OON Scorer: Adjust out-of-network content scores

Selection

Sort by final score and select top K candidates

Post-Selection Processing

Final validation and cleanup:

Visibility filtering (spam, violence, gore detection)
Conversation deduplication

Side Effects

Cache request information for future optimizations

Implementation

The pipeline is constructed using the builder pattern, assembling all components:

home-mixer/candidate_pipeline/phoenix_candidate_pipeline.rs

impl PhoenixCandidatePipeline {
    async fn build_with_clients(
        uas_fetcher: Arc<UserActionSequenceFetcher>,
        phoenix_client: Arc<dyn PhoenixPredictionClient>,
        phoenix_retrieval_client: Arc<dyn PhoenixRetrievalClient>,
        thunder_client: Arc<ThunderClient>,
        // ... other clients
    ) -> PhoenixCandidatePipeline {
        // Query Hydrators
        let query_hydrators = vec![
            Box::new(UserActionSeqQueryHydrator::new(uas_fetcher)),
            Box::new(UserFeaturesQueryHydrator { strato_client }),
        ];

        // Sources
        let sources = vec![
            Box::new(PhoenixSource { phoenix_retrieval_client }),
            Box::new(ThunderSource { thunder_client }),
        ];

        // ... filters, scorers, selectors
    }
}

Request Flow

When a user requests their For You feed:

home-mixer/server.rs

#[tonic::async_trait]
impl ScoredPostsService for HomeMixerServer {
    async fn get_scored_posts(
        &self,
        request: Request<ScoredPostsQuery>,
    ) -> Result<Response<ScoredPostsResponse>, Status> {
        let query = ScoredPostsQuery::new(
            proto_query.viewer_id,
            proto_query.client_app_id,
            proto_query.country_code,
            proto_query.language_code,
            proto_query.seen_ids,
            proto_query.served_ids,
            proto_query.in_network_only,
            // ...
        );
        
        let pipeline_result = self.phx_candidate_pipeline.execute(query).await;
        
        let scored_posts: Vec<ScoredPost> = pipeline_result
            .selected_candidates
            .into_iter()
            .map(|candidate| ScoredPost { /* ... */ })
            .collect();
            
        Ok(Response::new(ScoredPostsResponse { scored_posts }))
    }
}

Key Components

Sources

Two primary candidate sources run in parallel:

Thunder Source

Retrieves recent posts from accounts the user follows (in-network)

Phoenix Source

ML-based retrieval of relevant posts from the global corpus (out-of-network)

Filters

Home Mixer applies extensive filtering to ensure feed quality:

Filter	Purpose
`DropDuplicatesFilter`	Remove duplicate post IDs
`CoreDataHydrationFilter`	Remove posts missing core metadata
`AgeFilter`	Remove posts older than threshold
`SelfTweetFilter`	Remove user’s own posts
`RetweetDeduplicationFilter`	Dedupe reposts of same content
`IneligibleSubscriptionFilter`	Remove paywalled content user can’t access
`PreviouslySeenPostsFilter`	Remove already seen posts
`PreviouslyServedPostsFilter`	Remove posts served in current session
`MutedKeywordFilter`	Remove posts with muted keywords
`AuthorSocialgraphFilter`	Remove blocked/muted authors
`VFFilter`	Remove spam/violence/gore
`DedupConversationFilter`	Deduplicate conversation threads

Scorers

Multiple scorers combine to produce the final relevance score:

Phoenix Scorer

Calls the Phoenix ranking model to get ML predictions for each engagement type (like, reply, repost, etc.)

Weighted Scorer

Combines multiple predictions into a single relevance score using weighted sum

Author Diversity Scorer

Attenuates scores for posts from the same author to ensure feed diversity

OON Scorer

Adjusts scores for out-of-network content based on user preferences

Configuration

Key parameters controlling pipeline behavior:

// Maximum results from each source
pub const THUNDER_MAX_RESULTS: u32 = 1000;
pub const PHOENIX_MAX_RESULTS: usize = 500;

// Final result size
pub const RESULT_SIZE: usize = 100;

// Post age limits
pub const MAX_POST_AGE: u64 = 86400 * 2; // 2 days

Performance Characteristics

The pipeline is highly optimized for low latency:

Parallel execution: Sources and hydrators run concurrently
Sub-millisecond lookups: Thunder provides in-memory access to recent posts
Batch prediction: Phoenix scores all candidates in a single inference call
Streaming responses: gRPC with Zstd compression for efficient transport

Typical request latency: 100-300ms end-to-end

Monitoring

Home Mixer emits detailed metrics at each pipeline stage:

#[xai_stats_macro::receive_stats]
async fn get_scored_posts(...) -> Result<...> {
    // Automatic instrumentation tracks:
    // - Request count and latency
    // - Candidates from each source
    // - Filtered vs kept candidates
    // - Final result size
}

Candidate Pipeline

Framework that powers Home Mixer’s orchestration

Phoenix

ML component for retrieval and ranking

Thunder

In-memory post store for in-network content

Example Request

grpcurl -d '{
  "viewer_id": 12345,
  "client_app_id": "web",
  "country_code": "US",
  "language_code": "en",
  "seen_ids": [],
  "served_ids": [],
  "in_network_only": false
}' \
  home-mixer.example.com:443 \
  xai.ScoredPostsService/GetScoredPosts

Set in_network_only: true to disable Phoenix retrieval and only return posts from followed accounts.

Getting Started

Core Components

How It Works

Phoenix ML System

Implementation

Key Concepts

Overview

Architecture

Pipeline Stages

Implementation

Request Flow

Key Components

Sources

Thunder Source

Phoenix Source

Filters

Scorers

Configuration

Performance Characteristics

Monitoring

Candidate Pipeline

Phoenix

Thunder

Example Request

​Overview

​Architecture

​Pipeline Stages

​Implementation

​Request Flow

​Key Components

​Sources

Thunder Source

Phoenix Source

​Filters

​Scorers

​Configuration

​Performance Characteristics

​Monitoring

​Related Components

Candidate Pipeline

Phoenix

Thunder

​Example Request

Overview

Architecture

Pipeline Stages

Implementation

Request Flow

Key Components

Sources

Filters

Scorers

Configuration

Performance Characteristics

Monitoring

Related Components

Example Request