Skip to main content

Documentation Index

Fetch the complete documentation index at: https://mintlify.com/vruizz22/innova-backend-serverless/llms.txt

Use this file to discover all available pages before exploring further.

The rule engine is Innova’s first layer of error classification. Every student attempt passes through it before anything else — no network call, no AI inference. The design is grounded in Brown & VanLehn’s (1980) Repair Theory, which established that procedural arithmetic errors are not random: students apply consistent but faulty procedures that can be catalogued. By encoding those known “bugs” as deterministic rules, Innova can classify the majority of errors instantly and with high confidence, reserving the more expensive LLM path for genuinely ambiguous cases.

​
Design

The rule engine is built around the Strategy + Factory pattern.
  • One TypeScript class per math subdomain implements the RuleEngineStrategy interface.
  • RuleEngineFactory maintains a registry keyed by subdomain code (ARITH_SUB, FRACT_MUL, etc.) and resolves the right strategy at runtime.
  • RuleEngineService is the NestJS injectable that orchestrates the call: it receives the subdomain code alongside the CreateAttemptDto payload and delegates to the resolved strategy.
// engine.service.ts
@Injectable()
export class RuleEngineService {
  constructor(private readonly factory: RuleEngineFactory) {}

  classify(
    payload: CreateAttemptDto,
    subdomainCode: string,
  ): RuleClassificationResult {
    const strategy = this.factory.getStrategy(subdomainCode);
    return strategy.classify(payload);
  }
}
This keeps strategies fully decoupled: adding support for a new subdomain means creating one file and registering one entry in the factory — nothing else changes.

​
Supported Topics

The factory registry maps 19 subdomain codes to their concrete strategy classes across 7 math domains.
Subdomain codeStrategy classExample error types
ARITH_ADDAdditionCarryStrategyARITH_ADD_CARRY_OMITTED_G3, ARITH_ADD_CARRY_WRONG_COLUMN_G3
ARITH_SUBSubtractionBorrowStrategyARITH_SUB_BORROW_OMITTED_TENS_G3, ARITH_SUB_BORROW_FROM_ZERO_G3
ARITH_MULMultiplicationStrategyARITH_MUL_ZERO_TIMES_N_EQUALS_N_G3, ARITH_MUL_PARTIAL_NOT_SHIFTED_G5
ARITH_DIVDivisionLongStrategyARITH_DIV_DIVISOR_DIVIDEND_SWAPPED_G4, ARITH_DIV_QUOTIENT_ZERO_SKIPPED_G5
INT_ADD / INT_SUB / INT_MULIntAdditionStrategy, IntSubtractionStrategy, IntMultiplicationStrategyINT_MUL_NEG_TIMES_NEG_NEG_G7, INT_ADD_DIFF_SIGN_ADDS_MAGNITUDES_G7
FRACT_ADDSUBFractionSameDenomStrategyFRACT_ADDSUB_SAME_DENOM_ADDS_DENOM_G5, FRACT_ADDSUB_LCD_WRONG_G6
FRACT_MULFractionMultiplicationStrategyFRACT_MUL_CROSS_MULTIPLIES_G6, FRACT_MUL_SEEKS_COMMON_DENOM_G6
FRACT_DIVFractionDivisionStrategyFRACT_DIV_NO_RECIPROCAL_G7, FRACT_DIV_INVERTS_FIRST_FRACTION_G7
DEC_ADD / DEC_SUB / DEC_MUL / DEC_DIVDecimalAdditionStrategy, DecimalSubtractionStrategy, DecimalMultiplicationStrategy, DecimalDivisionStrategyDEC_ADD_RIGHT_ALIGNED_LIKE_INTEGERS_G5, DEC_MUL_POINT_PLACEMENT_ERROR_G6
RATIO_PERCENT / RATIO_PROPORTIONPercentStrategy, ProportionStrategyPercent and proportion procedure errors
ALGEBRA_EQ_LINEARLinearEquationStrategyLinear equation solving errors
POW_POWER / POW_ROOTPowerLawsStrategy, RootLawsStrategyExponent and root law errors

​
Error Taxonomy

Error codes follow a structured naming convention: <DOMAIN>_<SUBDOMAIN>_<BUG_NAME>_<GRADE_LEVEL>. For example:
  • ARITH_SUB_BORROW_OMITTED_TENS_G3 — Grade 3 subtraction, borrow omitted in the tens column
  • ARITH_ADD_CARRY_WRONG_COLUMN_G3 — Grade 3 addition, carry added to the wrong column
  • ARITH_MUL_PARTIAL_NOT_SHIFTED_G5 — Grade 5 multiplication, second partial product not shifted
  • ARITH_DIV_QUOTIENT_ZERO_SKIPPED_G5 — Grade 5 long division, interior zero dropped from quotient
  • FRACT_DIV_NO_RECIPROCAL_G7 — Grade 7 fraction division, divisor not inverted
  • DEC_MUL_POINT_PLACEMENT_ERROR_G6 — Grade 6 decimal multiplication, decimal point misplaced
Cross-domain transversal codes (no grade suffix) cover errors that appear across operations:
  • ARITH_TRANSV_DIGIT_TRANSPOSITION — digits of the answer are a permutation of the correct answer
  • ARITH_TRANSV_PLACE_VALUE_ERROR — answer is a factor-of-10 shift of the correct answer
  • ARITH_TRANSV_FACT_ERROR — basic arithmetic fact recall error (off by ≤ 2)
The full auto-generated catalog (error-tags.generated.ts) contains 2 624 active tags across all domains. The rule engine targets a ≥ 75% classification rate on in-scope subdomains; the remainder fall through to UNCLASSIFIED.

​
UNCLASSIFIED Fallback

When no deterministic rule matches — or when the subdomain code has no registered strategy — the factory returns a built-in fallback: 
const UNCLASSIFIED_FALLBACK: RuleEngineStrategy = {
  subdomainCode: '__FALLBACK__',
  classify: () => ({
    isCorrect: false,
    errorType: 'UNCLASSIFIED',
    confidence: 0,
  }),
};
An attempt tagged UNCLASSIFIED is enqueued to SQS for asynchronous LLM classification. The two-tier architecture means deterministic classification remains the fast path (< 5 ms, in-process), while AI inference handles only the tail that rules cannot cover.

​
Strategy Interface

Every strategy implements this interface: 
// strategy.interface.ts
import { type CreateAttemptDto } from '@modules/attempts/dto/create-attempt.dto';

// errorType will be typed as ErrorTagCode (from error-tags.generated.ts) once codegen runs.
// Until then, it's a string constrained to valid v8 naming convention codes.
export interface RuleClassificationResult {
  isCorrect: boolean;
  errorType: string;
  confidence: number;
  evidence?: string[];
}

export interface RuleEngineStrategy {
  readonly subdomainCode: string;
  classify(payload: CreateAttemptDto): RuleClassificationResult;
}
The evidence array is optional but strongly encouraged — it is logged and surfaced in the admin UI to help validate rule quality.

​
Example: Subtraction Borrow Strategy

The SubtractionBorrowStrategy implements eight deterministic rules in priority order. Here is an excerpt showing the first four: 
// subtraction-borrow.strategy.ts (excerpt)
export class SubtractionBorrowStrategy implements RuleEngineStrategy {
  readonly subdomainCode = 'ARITH_SUB';

  classify(payload: CreateAttemptDto): RuleClassificationResult {
    const { expectedAnswer, studentAnswer, rawSteps } = payload;
    const minuend = payload.minuend ?? 0;
    const subtrahend = payload.subtrahend ?? 0;

    if (studentAnswer === expectedAnswer) {
      return { isCorrect: true, errorType: 'CORRECT', confidence: 1.0 };
    }

    // Rule 1: minuend and subtrahend swapped
    if (subtrahend > minuend && studentAnswer === subtrahend - minuend) {
      return {
        isCorrect: false,
        errorType: 'ARITH_SUB_MINUEND_SUBTRAHEND_SWAPPED_G3',
        confidence: 0.95,
        evidence: [
          `Expected ${minuend}-${subtrahend}=${expectedAnswer}; student got ${subtrahend}-${minuend}=${studentAnswer}`,
        ],
      };
    }

    // Rule 2: borrow omitted — subtracted each column without borrowing
    const noBorrowResult = computeNoBorrowResult(minuend, subtrahend);
    const unitsM = minuend % 10;
    const unitsS = subtrahend % 10;
    const tensM = Math.floor(minuend / 10) % 10;
    if (studentAnswer === noBorrowResult && noBorrowResult !== expectedAnswer) {
      if (unitsS > unitsM && tensM > 0) {
        return {
          isCorrect: false,
          errorType: 'ARITH_SUB_BORROW_OMITTED_TENS_G3',
          confidence: 0.93,
          evidence: [
            `Units column: ${unitsM}-${unitsS} done without borrow → answer ${studentAnswer} vs expected ${expectedAnswer}`,
          ],
        };
      }
    }

    // Rule 3: borrow omitted at hundreds column
    const hundredsM = Math.floor(minuend / 100) % 10;
    const hundredsS = Math.floor(subtrahend / 100) % 10;
    const tensForHundreds = Math.floor(minuend / 10) % 10;
    if (
      hundredsS > hundredsM &&
      tensForHundreds === 0 &&
      studentAnswer === noBorrowResult
    ) {
      return {
        isCorrect: false,
        errorType: 'ARITH_SUB_BORROW_OMITTED_HUNDREDS_G3',
        confidence: 0.91,
        evidence: [`Hundreds column ${hundredsM}-${hundredsS} without borrow`],
      };
    }

    // Rule 4: borrowing from zero — tens is 0, hundreds exist, borrow propagation failed
    if (tensM === 0 && hundredsM > 0 && unitsS > unitsM) {
      return {
        isCorrect: false,
        errorType: 'ARITH_SUB_BORROW_FROM_ZERO_G3',
        confidence: 0.87,
        evidence: [`Borrow propagation through zero in tens column failed`],
      };
    }

    return {
      isCorrect: false,
      errorType: 'UNCLASSIFIED',
      confidence: 0.0,
      evidence: ['No deterministic rule matched'],
    };
  }
}

​
How to Add a Strategy

1

Implement RuleEngineStrategy

Create a new file in src/modules/attempts/rule-engine/strategies/. Export a class that implements RuleEngineStrategy, setting subdomainCode to the new subdomain key and writing classify() as a series of prioritized if guards.
2

Register in the factory

Open factory.ts, import your class, and add an entry to the REGISTRY object:
MY_NEW_CODE: new MyNewStrategy(),
3

Write unit tests

Add a spec file beside the strategy. Cover at least: correct answer returns isCorrect: true; each documented error rule returns the right errorType and a confidence ≥ 0.75; the fallback returns UNCLASSIFIED with confidence: 0.
4

Update error-tags.generated.ts

Run pnpm tsx scripts/codegen-error-tags.ts to regenerate the ErrorTagCode enum with any new tags you introduced.
The rule engine runs entirely in-process — no database queries, no HTTP calls. End-to-end classification time is consistently under 5 ms per attempt. This is by design: strategies must remain pure functions of the CreateAttemptDto payload.

Build docs developers (and LLMs) love

Get started for freeTalk to us