Red Flag Indicators Overview

What Are Red Flag Indicators?

Cardinal calculates 11 procurement indicators designed to identify potential red flags in contracting processes. These indicators detect patterns that may suggest corruption, collusion, or other irregularities in public procurement, such as:

Bid manipulation and collusion between tenderers
Suspicious pricing patterns
Irregular disqualification of bids
Procedural irregularities

Red flags are organized into four groups based on what entity they analyze:

Group	Description	Identifier Field
OCID	Contracting process indicators	`/ocid`
Buyer	Buyer behavior indicators	`/buyer/id`
ProcuringEntity	Procuring entity behavior indicators	`/tender/procuringEntity/id`
Tenderer	Tenderer behavior indicators	`/bids/details[]/tenderers[]/id`

Running the Indicators Command

To calculate indicators, use the indicators command with your prepared OCDS data:

ocdscardinal indicators --settings settings.ini prepared.jsonl > results.json

Before running indicators, ensure you’ve prepared your data with the prepare command to correct quality issues and standardize the format.

Command Options

--settings

string

required

Path to the settings file containing indicator configuration

--verbose

flag

Increase verbosity level (use multiple times: -v, -vv, -vvv)

--count

flag

Print the number of results per group to standard error

--no-meta

flag

Exclude the Meta key from the results object

--map

flag

Include the Maps key, mapping contracting processes to organizations

Understanding the Output Format

The indicators command produces JSON output organized hierarchically by group, identifier, indicator code, and value.

Basic Structure

{
  "OCID": {
    "ocds-6550wx-JRFPFA-DAF-CM-2021-0012": {
      "R036": 1.0
    }
  },
  "Buyer": {
    "DO-RPE-55216": {
      "R038": 0.8
    }
  },
  "ProcuringEntity": {},
  "Tenderer": {},
  "Meta": {
    "R024": {
      "q1": 66.6667,
      "q3": 100.0,
      "lower_fence": 16.6667
    }
  }
}

Output Components

Top-level groups

The first level contains four keys representing different entity types: OCID, Buyer, ProcuringEntity, and Tenderer.

Identifiers

Within each group, keys are identifiers extracted from your OCDS data (e.g., contracting process IDs, organization IDs).

Indicator codes

Within each identifier, keys are indicator codes (e.g., R003, R024) that raised a red flag for that entity.

Values

Each indicator code maps to a numeric value. The meaning depends on the indicator:

Binary flags: 1.0 indicates the red flag is present
Ratios/percentages: Values between 0.0 and 1.0
Counts: Integer values as decimals

Metadata Section

The Meta key provides statistical information about indicators that use outlier detection:

{
  "Meta": {
    "R024": {
      "q1": 66.6667,        // First quartile
      "q3": 100.0,          // Third quartile
      "lower_fence": 16.6667 // Lower fence for outlier detection
    }
  }
}

Use --no-meta to exclude metadata if you only need the indicator results.

Interpreting Results

When an Indicator Appears

An indicator code appears in the output only if a red flag was raised. If an indicator didn’t produce a result — either because it couldn’t be calculated or no red flag was detected — the code won’t appear.

Reading Values

The output value depends on the indicator type:

Binary Indicators
Ratio Indicators
Count Indicators

Value is always 1.0 when present, indicating the red flag exists.Example: R035 (All except winning bid disqualified), R036 (Lowest bid disqualified)

{
  "OCID": {
    "ocds-123": {
      "R036": 1.0  // Red flag present
    }
  }
}

Value between 0.0 and 1.0 representing a percentage or ratio.Example: R025 (Ratio of winning bids to submitted bids), R038 (Ratio of disqualified bids)

{
  "Tenderer": {
    "SUPPLIER-001": {
      "R025": 0.15  // Only 15% of bids won
    }
  }
}

Numeric count as a decimal value.Example: R028 (Number of bids with identical prices), R048 (Variety of items supplied)

{
  "OCID": {
    "ocds-456": {
      "R028": 3.0  // 3 bids with same price
    }
  }
}

Configuration Overview

All indicators can be configured in the settings.ini file. Configuration is optional — Cardinal provides sensible defaults.

Enabling an Indicator

To enable an indicator, create a section with its code:

[R024]

You don’t need to set any properties unless you want to customize behavior.

Disabling an Indicator

To disable an indicator, remove or comment out its section:

; [R024]
; threshold = 0.05

Global Settings

Some settings apply to all indicators:

# Main currency for monetary comparisons
currency = USD

# Exclude procedures globally
[exclusions]
procurement_method_details = Random Selection|Sorteo de Obras

# Exclude procedures from price comparisons only
no_price_comparison_procurement_methods = Random Selection|Sorteo de Obras

# Or, include only specific procedures in price comparisons
price_comparison_procurement_methods = Reverse Auction

Each indicator’s documentation page describes its individual configuration options and defaults.

All Red Flag Indicators

Cardinal includes 11 red flag indicators that detect various suspicious patterns:

R003: Short Submission Period

Contracting processes with submission periods that are too short

R018: Single Bid Received

Only one tenderer submitted a bid

R024: Price Close to Winning Bid

Second-lowest bid is suspiciously close to the winning bid (low outlier)

R025: Excessive Unsuccessful Bids

Tenderer has an unusually low ratio of winning bids (low outlier)

R028: Identical Bid Prices

Different tenderers submitted bids with the same price

R030: Late Bid Won

The winning bid was received after the submission deadline

R035: All Except Winner Disqualified

All bids are disqualified except those from the winner

R036: Lowest Bid Disqualified

The lowest bid is disqualified when award criterion is price only

R038: Excessive Disqualified Bids

High ratio of disqualified bids per buyer, procuring entity, or tenderer (high outlier)

R048: Heterogeneous Supplier

Tenderer supplies an unusually wide variety of items (high outlier)

R058: Heavily Discounted Bid

Winning bid is significantly lower than second-lowest bid (high outlier)

Indicator Summary Table

Code	Title	Group	Type	Description
R003	Short submission period	OCID	Threshold	Submission period is too short
R018	Single bid received	OCID	Binary	Only one tenderer submitted a bid
R024	Price close to winning bid	OCID	Outlier	Second-lowest bid too close to winner (low)
R025	Excessive unsuccessful bids	Tenderer	Outlier	Low ratio of winning bids (low)
R028	Identical bid prices	OCID	Count	Different tenderers with same price
R030	Late bid won	OCID	Binary	Winner received after deadline
R035	All except winning bid disqualified	OCID	Binary	Only winner’s bid not disqualified
R036	Lowest bid disqualified	OCID	Binary	Lowest bid disqualified, price-only award
R038	Excessive disqualified bids	Buyer, ProcuringEntity, Tenderer	Outlier	High ratio of disqualified bids (high)
R048	Heterogeneous supplier	Tenderer	Outlier	Wide variety of items supplied (high)
R058	Heavily discounted bid	OCID	Outlier	Winner much lower than second bid (high)

Common Exclusions

All indicators exclude contracting processes where:

Invalid or missing data

The ocid is not a string
Relevant organization references don’t set an id
Required fields are missing or malformed

Cancelled contracting process

The /tender/status is cancelled

Currency mismatch

Monetary values don’t use the main currency
Configure the main currency with: currency = USD
Otherwise, the first observed currency is used

Global exclusions

Procedures matching excluded procurement_method_details
Configured in the [exclusions] section

Workflow

Follow these steps to analyze your procurement data:

Select indicators

Enable or disable indicators in your settings.ini file. If you used the init command, all indicators are enabled by default.

Run the command

Execute the indicators command with your prepared data:

ocdscardinal indicators --settings settings.ini prepared.jsonl > results.json

Review the results

Examine the JSON output to identify flagged entities. Cross-reference with your source data to investigate potential issues.

Adjust settings

Modify indicator thresholds and configuration to reduce false positives based on your review. Iterate until satisfied with the results.

Next Steps

Configure Settings

Learn about all configuration options and how to customize indicators

Explore Indicators

Read detailed documentation for each individual red flag indicator

Commands Reference

View full command-line options and examples

Prepare Data

Ensure your data is properly prepared before running indicators

Need help? Contact James McKinney, OCP’s Head of Technology, with questions or feedback.

Overview

Indicators

Documentation Index

​What Are Red Flag Indicators?

​Running the Indicators Command

​Command Options

​Understanding the Output Format

​Basic Structure

​Output Components

​Metadata Section

​Interpreting Results

​When an Indicator Appears

​Reading Values

​Configuration Overview

​Enabling an Indicator

​Disabling an Indicator

​Global Settings

​All Red Flag Indicators

R003: Short Submission Period

R018: Single Bid Received

R024: Price Close to Winning Bid

R025: Excessive Unsuccessful Bids

R028: Identical Bid Prices

R030: Late Bid Won

R035: All Except Winner Disqualified

R036: Lowest Bid Disqualified

R038: Excessive Disqualified Bids

R048: Heterogeneous Supplier

R058: Heavily Discounted Bid

​Indicator Summary Table

​Common Exclusions

​Workflow

​Next Steps

Configure Settings

Explore Indicators

Commands Reference

Prepare Data

Build docs developers (and LLMs) love

What Are Red Flag Indicators?

Running the Indicators Command

Command Options

Understanding the Output Format

Basic Structure

Output Components

Metadata Section

Interpreting Results

When an Indicator Appears

Reading Values

Configuration Overview

Enabling an Indicator

Disabling an Indicator

Global Settings

All Red Flag Indicators

Indicator Summary Table

Common Exclusions

Workflow

Next Steps