Scorers

Overview

Scorers compute prediction scores and ranking signals for candidates. They run sequentially after filtering, allowing each scorer to see the results of previous scorers. Scorers update candidate fields with computed scores.

Scorer Trait

The Scorer trait defines the interface for scoring candidates:

candidate-pipeline/scorer.rs

#[async_trait]
pub trait Scorer<Q, C>: Send + Sync
where
    Q: Clone + Send + Sync + 'static,
    C: Clone + Send + Sync + 'static,
{
    /// Decide if this scorer should run for the given query
    fn enable(&self, _query: &Q) -> bool {
        true
    }

    /// Score candidates by performing async operations.
    /// Returns candidates with this scorer's fields populated.
    ///
    /// IMPORTANT: The returned vector must have the same candidates in the same order as the input.
    /// Dropping candidates in a scorer is not allowed - use a filter stage instead.
    async fn score(&self, query: &Q, candidates: &[C]) -> Result<Vec<C>, String>;

    /// Update a single candidate with the scored fields.
    /// Only the fields this scorer is responsible for should be copied.
    fn update(&self, candidate: &mut C, scored: C);

    /// Update all candidates with the scored fields from `scored`.
    /// Default implementation iterates and calls `update` for each pair.
    fn update_all(&self, candidates: &mut [C], scored: Vec<C>) {
        for (c, s) in candidates.iter_mut().zip(scored) {
            self.update(c, s);
        }
    }

    fn name(&self) -> &'static str {
        util::short_type_name(type_name_of_val(self))
    }
}

The score method must return candidates in the same order as the input. The returned vector must have the same length. Dropping or reordering candidates is not allowed - use a filter instead.

fn(&self, &Q) -> bool

Optional gating logic to conditionally enable/disable the scorer based on query parameters

async fn(&self, &Q, &[C]) -> Result<Vec<C>, String>

The main scoring method. Returns scored candidates with prediction fields populated.

fn(&self, &mut C, C)

Updates a single candidate with scored fields. Only copy fields this scorer is responsible for.

fn(&self) -> &'static str

Returns the scorer name for logging and metrics. Auto-generated from the type name by default.

Phoenix Scorer

The Phoenix scorer calls the Phoenix prediction service to compute engagement probabilities:

home-mixer/scorers/phoenix_scorer.rs

pub struct PhoenixScorer {
    pub phoenix_client: Arc<dyn PhoenixPredictionClient + Send + Sync>,
}

#[async_trait]
impl Scorer<ScoredPostsQuery, PostCandidate> for PhoenixScorer {
    async fn score(
        &self,
        query: &ScoredPostsQuery,
        candidates: &[PostCandidate],
    ) -> Result<Vec<PostCandidate>, String> {
        let user_id = query.user_id as u64;
        let prediction_request_id = request_util::generate_request_id();
        let last_scored_at_ms = Self::current_timestamp_millis();

        if let Some(sequence) = &query.user_action_sequence {
            let tweet_infos: Vec<xai_recsys_proto::TweetInfo> = candidates
                .iter()
                .map(|c| {
                    let tweet_id = c.retweeted_tweet_id.unwrap_or(c.tweet_id as u64);
                    let author_id = c.retweeted_user_id.unwrap_or(c.author_id);
                    xai_recsys_proto::TweetInfo {
                        tweet_id,
                        author_id,
                        ..Default::default()
                    }
                })
                .collect();

            let result = self
                .phoenix_client
                .predict(user_id, sequence.clone(), tweet_infos)
                .await;

            if let Ok(response) = result {
                let predictions_map = self.build_predictions_map(&response);

                let scored_candidates = candidates
                    .iter()
                    .map(|c| {
                        let lookup_tweet_id = c.retweeted_tweet_id.unwrap_or(c.tweet_id as u64);

                        let phoenix_scores = predictions_map
                            .get(&lookup_tweet_id)
                            .map(|preds| self.extract_phoenix_scores(preds))
                            .unwrap_or_default();

                        PostCandidate {
                            phoenix_scores,
                            prediction_request_id: Some(prediction_request_id),
                            last_scored_at_ms,
                            ..Default::default()
                        }
                    })
                    .collect();

                return Ok(scored_candidates);
            }
        }

        // Return candidates unchanged if no scoring could be done
        Ok(candidates.to_vec())
    }

    fn update(&self, candidate: &mut PostCandidate, scored: PostCandidate) {
        candidate.phoenix_scores = scored.phoenix_scores;
        candidate.prediction_request_id = scored.prediction_request_id;
        candidate.last_scored_at_ms = scored.last_scored_at_ms;
    }
}

Phoenix Scorer Features

Calls Phoenix prediction service with user action sequence
Computes engagement probabilities (like, reply, retweet, etc.)
Handles retweets by looking up scores for the original tweet
Includes timestamp and request ID for debugging
Gracefully degrades if prediction fails (returns candidates unscored)

Phoenix Scores

The Phoenix scorer populates these engagement prediction fields:

Option<f64>

Probability the user will like the tweet

Option<f64>

Probability the user will reply to the tweet

Option<f64>

Probability the user will retweet the tweet

Option<f64>

Probability the user will expand photos

Option<f64>

Probability the user will click the tweet

Option<f64>

Probability the user will click the author’s profile

Option<f64>

Probability the user will watch a video (Video Quality View)

Probability the user will share the tweet

Option<f64>

Probability the user will dwell on the tweet

Option<f64>

Expected dwell time in seconds (continuous action)

Option<f64>

Probability the user will mark “not interested” (negative signal)

Option<f64>

Probability the user will block the author (negative signal)

Option<f64>

Probability the user will mute the author (negative signal)

Option<f64>

Probability the user will report the tweet (negative signal)

Weighted Scorer

The Weighted scorer combines Phoenix scores into a single ranking score:

home-mixer/scorers/weighted_scorer.rs

pub struct WeightedScorer;

#[async_trait]
impl Scorer<ScoredPostsQuery, PostCandidate> for WeightedScorer {
    async fn score(
        &self,
        _query: &ScoredPostsQuery,
        candidates: &[PostCandidate],
    ) -> Result<Vec<PostCandidate>, String> {
        let scored = candidates
            .iter()
            .map(|c| {
                let weighted_score = Self::compute_weighted_score(c);
                let normalized_weighted_score = normalize_score(c, weighted_score);

                PostCandidate {
                    weighted_score: Some(normalized_weighted_score),
                    ..Default::default()
                }
            })
            .collect();

        Ok(scored)
    }

    fn update(&self, candidate: &mut PostCandidate, scored: PostCandidate) {
        candidate.weighted_score = scored.weighted_score;
    }
}

impl WeightedScorer {
    fn compute_weighted_score(candidate: &PostCandidate) -> f64 {
        let s: &PhoenixScores = &candidate.phoenix_scores;
        let vqv_weight = Self::vqv_weight_eligibility(candidate);

        let combined_score = Self::apply(s.favorite_score, p::FAVORITE_WEIGHT)
            + Self::apply(s.reply_score, p::REPLY_WEIGHT)
            + Self::apply(s.retweet_score, p::RETWEET_WEIGHT)
            + Self::apply(s.photo_expand_score, p::PHOTO_EXPAND_WEIGHT)
            + Self::apply(s.click_score, p::CLICK_WEIGHT)
            + Self::apply(s.profile_click_score, p::PROFILE_CLICK_WEIGHT)
            + Self::apply(s.vqv_score, vqv_weight)
            + Self::apply(s.share_score, p::SHARE_WEIGHT)
            + Self::apply(s.share_via_dm_score, p::SHARE_VIA_DM_WEIGHT)
            + Self::apply(s.share_via_copy_link_score, p::SHARE_VIA_COPY_LINK_WEIGHT)
            + Self::apply(s.dwell_score, p::DWELL_WEIGHT)
            + Self::apply(s.quote_score, p::QUOTE_WEIGHT)
            + Self::apply(s.quoted_click_score, p::QUOTED_CLICK_WEIGHT)
            + Self::apply(s.dwell_time, p::CONT_DWELL_TIME_WEIGHT)
            + Self::apply(s.follow_author_score, p::FOLLOW_AUTHOR_WEIGHT)
            + Self::apply(s.not_interested_score, p::NOT_INTERESTED_WEIGHT)
            + Self::apply(s.block_author_score, p::BLOCK_AUTHOR_WEIGHT)
            + Self::apply(s.mute_author_score, p::MUTE_AUTHOR_WEIGHT)
            + Self::apply(s.report_score, p::REPORT_WEIGHT);

        Self::offset_score(combined_score)
    }

    fn vqv_weight_eligibility(candidate: &PostCandidate) -> f64 {
        if candidate
            .video_duration_ms
            .is_some_and(|ms| ms > p::MIN_VIDEO_DURATION_MS)
        {
            p::VQV_WEIGHT
        } else {
            0.0
        }
    }
}

Weighted Scorer Features

Combines multiple engagement scores into a single ranking score
Applies configurable weights to each engagement type
Conditionally applies VQV weight based on video duration
Includes negative signals (blocks, reports, not interested)
Normalizes scores for consistent ranking

The Weighted scorer is the final ranking signal used by the selector to sort candidates. It combines positive engagement signals (likes, retweets) with negative signals (blocks, reports) to produce a single score.

Available Scorers

The home-mixer service includes these scorers:

PhoenixScorer

Computes ML-based engagement predictions

WeightedScorer

Combines scores with configurable weights

OONScorer

Out-of-network specific scoring adjustments

AuthorDiversityScorer

Scores based on author diversity

Sequential Execution

Scorers run sequentially, allowing each scorer to see previous scores:

candidate-pipeline/candidate_pipeline.rs

async fn score(&self, query: &Q, mut candidates: Vec<C>) -> Vec<C> {
    let request_id = query.request_id().to_string();
    let expected_len = candidates.len();
    
    for scorer in self.scorers().iter().filter(|s| s.enable(query)) {
        match scorer.score(query, &candidates).await {
            Ok(scored) => {
                if scored.len() == expected_len {
                    scorer.update_all(&mut candidates, scored);
                } else {
                    warn!(
                        "request_id={} stage={:?} component={} skipped: length_mismatch expected={} got={}",
                        request_id,
                        PipelineStage::Scorer,
                        scorer.name(),
                        expected_len,
                        scored.len()
                    );
                }
            }
            Err(err) => {
                error!(
                    "request_id={} stage={:?} component={} failed: {}",
                    request_id,
                    PipelineStage::Scorer,
                    scorer.name(),
                    err
                );
            }
        }
    }
    candidates
}

If a scorer fails or returns mismatched length, the error is logged and candidates remain unscored. This ensures partial failures don’t break the pipeline.

Error Handling

If a scorer fails or returns mismatched length, the error is logged and candidates remain unmodified. The pipeline continues with the existing scores.

Best Practices

Maintain Order

The scored vector must match the input order and length:

let scored_candidates = candidates
    .iter()
    .map(|c| {
        // Compute scores for each candidate in order
        PostCandidate {
            weighted_score: Some(compute_score(c)),
            ..Default::default()
        }
    })
    .collect();

Partial Updates

Only update fields this scorer is responsible for:

fn update(&self, candidate: &mut PostCandidate, scored: PostCandidate) {
    candidate.weighted_score = scored.weighted_score;
    // Don't touch other fields
}

Graceful Degradation

If scoring fails, return candidates unchanged instead of erroring:

if let Ok(response) = prediction_result {
    // Score candidates with predictions
    return Ok(scored_candidates);
}
// Return candidates unchanged if prediction failed
Ok(candidates.to_vec())

Batch Operations

Batch external calls to minimize RPC overhead:

let tweet_infos: Vec<_> = candidates.iter().map(|c| build_info(c)).collect();
let predictions = client.predict(user_id, sequence, tweet_infos).await;

Score Composition

Scorers typically run in this order:

Phoenix Scorer

Computes ML-based engagement predictions for all candidates

Weighted Scorer

Combines Phoenix scores into a single ranking score using configurable weights

Diversity Scorer

Applies diversity adjustments to promote variety in results

Score Normalization

The weighted scorer includes normalization to handle negative scores:

home-mixer/scorers/weighted_scorer.rs

fn offset_score(combined_score: f64) -> f64 {
    if p::WEIGHTS_SUM == 0.0 {
        combined_score.max(0.0)
    } else if combined_score < 0.0 {
        (combined_score + p::NEGATIVE_WEIGHTS_SUM) / p::WEIGHTS_SUM * p::NEGATIVE_SCORES_OFFSET
    } else {
        combined_score + p::NEGATIVE_SCORES_OFFSET
    }
}

Negative signals (blocks, reports) have negative weights, which can make the combined score negative. The offset ensures all final scores are positive for ranking.

Pipeline Framework

Understand the overall pipeline execution flow

Phoenix Prediction

Deep dive into Phoenix prediction system

Filters

Filter candidates before scoring

Ranking

Learn about the ranking algorithm

Getting Started

Core Components

How It Works

Phoenix ML System

Implementation

Key Concepts

Overview

Scorer Trait

Phoenix Scorer

Phoenix Scorer Features

Phoenix Scores

Weighted Scorer

Weighted Scorer Features

Available Scorers

PhoenixScorer

WeightedScorer

OONScorer

AuthorDiversityScorer

Sequential Execution

Error Handling

Best Practices

Score Composition

Score Normalization

Pipeline Framework

Phoenix Prediction

Filters

Ranking

​Overview

​Scorer Trait

​Phoenix Scorer

​Phoenix Scorer Features

​Phoenix Scores

​Weighted Scorer

​Weighted Scorer Features

​Available Scorers

PhoenixScorer

WeightedScorer

OONScorer

AuthorDiversityScorer

​Sequential Execution

​Error Handling

​Best Practices

​Score Composition

​Score Normalization

​Related Topics

Pipeline Framework

Phoenix Prediction

Filters

Ranking

Overview

Scorer Trait

Phoenix Scorer

Phoenix Scorer Features

Phoenix Scores

Weighted Scorer

Weighted Scorer Features

Available Scorers

Sequential Execution

Error Handling

Best Practices

Score Composition

Score Normalization

Related Topics