Skip to main content

​
Overview

Pushservice is the main push recommendation service at Twitter, responsible for generating recommendation-based notifications for users. It determines when to send push notifications, what content to recommend, and manages the delivery to users.

​
Core Functionalities

Pushservice provides two main handlers:

RefreshForPushHandler

Determines whether to send a recommendation push to a user and generates the best push recommendation item

SendHandler

Manages whether to send the push based on target user details and provided recommendation item

​
RefreshForPushHandler Pipeline

The primary handler for generating push notifications follows a comprehensive pipeline:
1

Build Target & Check Eligibility

  • Build a target user object based on the given user ID
  • Perform target-level filtering to determine if the user is eligible for a recommendation push
  • Check user notification settings, fatigue, and activity patterns
2

Fetch Candidates

Retrieve a list of potential candidates for the push by querying various candidate sources:
  • Tweet recommendations
  • Account recommendations
  • Trending topics
  • Event-based notifications
3

Candidate Hydration

Hydrate candidate details with batch calls to different downstream services:
  • Tweet metadata from Tweetypie
  • User information from User Service
  • Engagement data from various sources
4

Pre-Rank Filtering (Light Filtering)

Filter hydrated candidates with lightweight RPC calls:
  • Basic quality checks
  • User preference filters
  • Duplicate detection
5

Ranking

Multi-stage ranking process:
  • Feature hydration for candidates and target user
  • Light ranking to reduce candidate pool
  • Heavy ranking using ML models to predict engagement
6

Take Step (Heavy Filtering)

Take top-ranked candidates one by one and apply heavy filtering until one candidate passes all filter steps:
  • Advanced safety checks
  • Fatigue management
  • Notification quality thresholds
7

Send

Call the appropriate downstream service to deliver the eligible candidate as a push and in-app notification to the target user

​
Handler Implementation

class RefreshForPushHandler(
  val pushTargetUserBuilder: PushTargetUserBuilder,
  val candSourceGenerator: PushCandidateSourceGenerator,
  rfphRanker: RFPHRanker,
  candidateHydrator: PushCandidateHydrator,
  candidateValidator: RFPHCandidateValidator,
  rfphTakeStepUtil: RFPHTakeStepUtil,
  rfphRestrictStep: RFPHRestrictStep,
  val rfphNotifier: RefreshForPushNotifier,
  rfphStatsRecorder: RFPHStatsRecorder,
  mrRequestScriberNode: String,
  rfphFeatureHydrator: RFPHFeatureHydrator,
  rfphPrerankFilter: RFPHPrerankFilter,
  rfphLightRanker: RFPHLightRanker
)(
  globalStats: StatsReceiver)
    extends FetchRankFlowWithHydratedCandidates[Target, RawCandidate, PushCandidate]
Source: pushservice/src/main/scala/com/twitter/frigate/pushservice/refresh_handler/RefreshForPushHandler.scala:33

​
SendHandler Pipeline

A simplified handler for sending pre-determined push recommendations:
1

Build Target

Build a target user object based on the given user ID
2

Candidate Hydration

Hydrate the candidate details with batch calls to different downstream services
3

Feature Hydration

Perform feature hydration for candidates and target user
4

Heavy Filtering

Perform filtering and validation checking for the given candidate
5

Send

Call the appropriate downstream service to deliver the candidate as a push and/or in-app notification

​
Candidate Sources

Pushservice integrates with multiple candidate sources through adaptors:

​
Tweet Candidates

  • ContentRecommenderMixerAdaptor - CR-Mixer tweet recommendations
  • EarlyBirdFirstDegreeCandidateAdaptor - Search index tweet candidates
  • FRSTweetCandidateAdaptor - Follow Recommendations Service tweet authors
  • HighQualityTweetsAdaptor - Curated high-quality content
  • ExploreVideoTweetCandidateAdaptor - Video content from Explore

​
Non-Tweet Candidates

  • ListsToRecommendCandidateAdaptor - List recommendations
  • GenericCandidateAdaptor - Generic notification types
  • LoggedOutPushCandidateSourceGenerator - Logged-out user notifications
Adaptors are located at: pushservice/src/main/scala/com/twitter/frigate/pushservice/adaptor/

​
Ranking Models

Pushservice uses a two-stage ranking approach:

​
Light Ranking

Fast, lightweight models to quickly filter candidates:
  • Low-latency feature set (< 100 features)
  • Simple models (logistic regression, small trees)
  • Quick engagement predictions
  • Target: p99 latency < 50ms

​
Heavy Ranking

Deep learning models for precise engagement prediction:
  • Comprehensive feature set (~1000+ features)
  • Complex neural networks
  • Detailed engagement probability scores
  • Multiple prediction targets:
    • Click probability
    • Positive engagement probability
    • Negative action probability
Ranking model implementations:
  • Light ranking: pushservice/src/main/python/models/light_ranking/
  • Heavy ranking: pushservice/src/main/python/models/heavy_ranking/

​
Filtering Strategy

​
Pre-Rank Filters (Light)

Applied to all candidates before ranking: 
class RFPHPrerankFilter(
  // Filter configurations
) {
  def apply(candidates: Seq[Candidate]): Seq[Candidate] = {
    candidates
      .filter(isNotDuplicate)
      .filter(meetsBasicQuality)
      .filter(userHasNotOptedOut)
      .filter(isWithinTimeWindow)
  }
}

​
Post-Rank Filters (Heavy)

Applied to top-ranked candidates:
  • Advanced safety and quality checks
  • Fatigue management (frequency capping)
  • User preference validation
  • Time-of-day optimization
  • Device and platform checks
The Take Step applies heavy filters sequentially to top candidates until one passes all checks. This ensures at most one notification is sent per refresh cycle.

​
Notification Types

Pushservice supports various notification types:

Tweet Recommendations

Tweets the user might be interested in

Account Recommendations

Suggested accounts to follow

Engagement Notifications

Likes, retweets, mentions from network

Trending Topics

Trending conversations and events

Lists

Recommended lists to join or follow

Spaces

Live audio conversations

​
Eligibility Checks

Before generating candidates, Pushservice validates target eligibility:

​
User-Level Checks

  • Notification settings and preferences
  • Push enabled for device
  • Not in quiet hours
  • Below daily notification quota
  • Sufficient time since last notification

​
Account-Level Checks

  • Account in good standing
  • Not suspended or restricted
  • Email/phone verified (for certain notification types)
  • Meets minimum activity threshold

​
Feature Hydration

Comprehensive feature extraction for ranking:

​
User Features

  • Engagement history and patterns
  • Notification response rates
  • Active hours and timezone
  • Language and location
  • Account age and follower count

​
Candidate Features

  • Tweet/account engagement metrics
  • Recency and virality
  • Content type and media presence
  • Author reputation and quality

​
Contextual Features

  • Time of day and day of week
  • Device type and OS
  • Network connection quality
  • User’s current activity state

​
Performance Optimization

​
Batching Strategy

// Batch candidate hydration
val hydratedCandidates = candidateHydrator.hydrate(
  candidates = candidates,
  batchSize = 100,
  timeout = 200.milliseconds
)

​
Caching

  • Cache user eligibility checks
  • Cache frequently accessed features
  • Cache model predictions for similar candidates

​
Parallel Processing

  • Parallel candidate source fetching
  • Concurrent feature hydration
  • Parallel filter evaluation where possible

​
Monitoring and Analytics

​
Key Metrics

  • Send Rate: Notifications sent per eligible user
  • Click-Through Rate (CTR): User engagement with notifications
  • Opt-Out Rate: Users disabling notifications
  • Latency: Time from trigger to notification delivery
  • Candidate Funnel: Drop-off at each pipeline stage

​
Logging and Debugging

val rfphStatsRecorder: RFPHStatsRecorder = new RFPHStatsRecorder(
  statsReceiver = statsReceiver.scope("RefreshForPushHandler")
)

// Record metrics at each stage
rfphStatsRecorder.recordCandidates(candidates)
rfphStatsRecorder.recordFiltered(filtered)
rfphStatsRecorder.recordSent(sent)

Build docs developers (and LLMs) love

Get started for freeTalk to us