Skip to main content

​
Product Mixer

Product Mixer is a common service framework and set of libraries that make it easy to build, iterate on, and own product surface areas at X. It provides the foundation for building execution pipelines out of reusable components with built-in monitoring and observability.

​
Overview

Product Mixer enables teams to focus on business logic rather than infrastructure concerns by providing:
  • Core Libraries: Build execution pipelines from small, well-defined, reusable components
  • Service Framework: Common service skeleton for hosting Product Mixer products
  • Component Library: Shared library of components from the Product Mixer team and community contributors

​
Architecture

Product Mixer applications are built from two main concepts: Pipelines and Components.

​
Components

Components break business logic into separate, standardized, reusable, testable, and easily composable pieces. Each component has a well-defined abstraction and single responsibility. Component Types:
  • Candidate Sources: Fetch candidates from underlying services
  • Filters: Remove unwanted candidates based on criteria
  • Scorers: Score and rank candidates
  • Decorators: Add additional information to candidates
  • Gates: Control pipeline execution flow
  • Transformers: Modify candidate data

​
Pipelines

Pipelines are configuration files that specify which components to use and when. They make code execution transparent and maintainable. 
// Example Pipeline Configuration (simplified)
class ExamplePipeline extends RecommendationPipeline {
  override val candidatePipelines: Seq[CandidatePipeline] = Seq(
    InNetworkCandidatePipeline,
    OutOfNetworkCandidatePipeline
  )
  
  override val scoringPipeline: ScoringPipeline = 
    HeavyRankerScoringPipeline
    
  override val filters: Seq[Filter] = Seq(
    AuthorDiversityFilter,
    VisibilityFilter
  )
}

​
Pipeline Hierarchy

1

Product Pipeline

Entry point that selects which Mixer or Recommendation Pipeline to run for a given request
ProductPipeline
  ├── Request validation
  ├── Pipeline selection logic
  └── Route to appropriate pipeline
2

Mixer Pipeline

Combines results from multiple heterogeneous Candidate Pipelines (e.g., tweets, ads, users)
MixerPipeline
  ├── Multiple Candidate Pipelines (different types)
  ├── Marshalling to domain objects
  └── Transport object creation
3

Recommendation Pipeline

Scores and ranks results from homogeneous Candidate Pipelines, returning top-ranked items
RecommendationPipeline
  ├── Candidate Pipelines (same type)
  ├── Scoring Pipeline
  ├── Ranking logic
  └── Top-N selection
4

Candidate Pipeline

Fetches candidates from sources and performs basic operations (filtering, decoration, feature hydration)
CandidatePipeline
  ├── Candidate Source query
  ├── Filters
  ├── Decorators
  └── Feature hydration
5

Scoring Pipeline

Applies ML models to score candidates for ranking
ScoringPipeline
  ├── Feature hydration
  ├── Model inference
  └── Score assignment

​
Request Flow

​
Component Composition

Components are designed to be highly reusable across different pipelines: 
class EarlybirdCandidateSource extends CandidateSource {
  override def apply(
    request: Request
  ): Stitch[Seq[TweetCandidate]] = {
    // Query search index
    earlybirdClient.search(
      query = request.query,
      numResults = request.maxResults
    ).map(_.results.map(TweetCandidate(_)))
  }
}

​
Home Mixer Example

Home Mixer is X’s main service for constructing Home Timelines, built entirely on Product Mixer.

​
For You Timeline Pipeline Structure

Top-level product pipeline for the For You timeline
Main orchestration layer - mixes Tweets with ads and users
Fetches Tweet candidates
Main Tweet recommendation layerCandidate Pipelines:
  • ScoredTweetsInNetworkCandidatePipelineConfig - In-network tweets
  • ScoredTweetsTweetMixerCandidatePipelineConfig - Mixed sources
  • ScoredTweetsUtegCandidatePipelineConfig - UTEG graph traversal
  • ScoredTweetsFrsCandidatePipelineConfig - Follow recommendations
Scoring:
  • ScoredTweetsScoringPipelineConfig - Feature hydration and ML ranking
Additional Pipelines:
  • ForYouConversationServiceCandidatePipelineConfig - Backup reverse chron
  • ForYouAdsCandidatePipelineConfig - Ad candidates
  • ForYouWhoToFollowCandidatePipelineConfig - User recommendations

​
Following Timeline Pipeline Structure

FollowingProductPipelineConfig
  └── FollowingMixerPipelineConfig
      ├── FollowingEarlybirdCandidatePipelineConfig
      ├── ConversationServiceCandidatePipelineConfig
      ├── FollowingAdsCandidatePipelineConfig
      └── FollowingWhoToFollowCandidatePipelineConfig

​
Lists Timeline Pipeline Structure

ListTweetsProductPipelineConfig
  └── ListTweetsMixerPipelineConfig
      ├── ListTweetsTimelineServiceCandidatePipelineConfig
      ├── ConversationServiceCandidatePipelineConfig
      └── ListTweetsAdsCandidatePipelineConfig

​
Key Benefits

Reusability

Write components once, use across multiple pipelines and products

Testability

Small, well-defined components are easy to unit test in isolation

Maintainability

Clear pipeline structure makes code easy to understand and modify

Observability

Built-in monitoring and metrics for all pipeline stages

Type Safety

Scala’s type system catches errors at compile time

Composability

Mix and match components to create new products quickly

​
Component Library

Product Mixer includes a shared component library with pre-built components:
  • Candidate Sources: Earlybird, UTEG, FRS, Cr Mixer
  • Filters: Visibility, Author Diversity, Content Balance, Feedback Fatigue
  • Scorers: Light Ranker, Heavy Ranker, Engagement Predictor
  • Decorators: Social Context, Conversation Modules, Feedback Options
  • Marshalling: Timeline instructions, Transport format converters

​
Performance Characteristics

Product Mixer uses Stitch (X’s asynchronous programming library) for efficient concurrent execution and automatic batching of requests.
Execution Model:
  • Parallel execution of independent pipeline stages
  • Automatic request batching to downstream services
  • Configurable timeouts and fallbacks
  • Circuit breakers for resilience

​
Learn More

Candidate Generation

Learn about candidate sourcing strategies

Ranking Systems

Explore light and heavy ranking pipelines

Navi ML Serving

Understand the ML serving infrastructure

Build docs developers (and LLMs) love

Get started for freeTalk to us