Skip to main content

Documentation Index

Fetch the complete documentation index at: https://mintlify.com/ragaeeb/kokokor/llms.txt

Use this file to discover all available pages before exploring further.

​
Overview

Layout elements are structural components detected in documents that help Kokokor better understand the document’s organization. Kokokor supports two types of layout elements:
  1. Horizontal Lines - Used for footnote boundary detection
  2. Rectangles - Used for heading and boxed content detection

​
Horizontal Lines for Footnote Detection

Horizontal lines in documents often serve as visual separators between main content and footnotes. Kokokor uses these lines to automatically classify text below them as footnotes.

​
How It Works

Kokokor identifies the last horizontal line that isn’t contained within a rectangle, and marks all text below this line as footnotes. This helps in:
  • Separating main content from supplementary information
  • Proper document structure reconstruction
  • Academic and formal document processing

​
Basic Usage

import { reconstructParagraphs } from 'kokokor';

const horizontalLines = [
  { x: 100, y: 2800, width: 600, height: 2 }, // Footer separator line
];

const result = reconstructParagraphs({
  observations: ocrObservations,
  page: pageContext,
  layout: {
    horizontalLines
  }
});

// Check for footnotes
result.paragraphs.forEach(para => {
  if (para.isFootnote) {
    console.log('Footnote:', para.text);
  }
});

​
Finding the Last Horizontal Line

Kokokor automatically finds the last horizontal separator for footnote detection: 

const rectangles = [/* detected rectangles */];
const horizontalLines = [/* detected horizontal lines */];
​
rectangles
BoundingBox[]
required
Array of rectangle bounding boxes to exclude from horizontal line consideration
​
horizontalLines
BoundingBox[]
required
Array of horizontal line bounding boxes to analyze
​
pixelTolerance
number
default:5
Pixel tolerance for containment checking and filtering lines near the top edge

​
Rectangles for Heading Detection

Rectangles (boxes) in documents often highlight important content like headings, titles, or callouts. Kokokor automatically classifies text within rectangles as headings.

​
How It Works

Kokokor checks if text bounding boxes are contained within detected rectangles (with tolerance). Text inside rectangles receives the isHeading flag and gets special formatting:
  • Headings are followed by blank lines in formatted output
  • Helps with document outline generation
  • Assists in hierarchical content structuring

​
Basic Usage

import { reconstructParagraphs } from 'kokokor';

const rectangles = [
  { x: 200, y: 100, width: 400, height: 50 }, // Title box
  { x: 150, y: 500, width: 500, height: 40 }, // Section heading box
];

const result = reconstructParagraphs({
  observations: ocrObservations,
  page: pageContext,
  layout: {
    rectangles
  }
});

// Check for headings
result.paragraphs.forEach(para => {
  if (para.isHeading) {
    console.log('Heading:', para.text);
  }
});

​
Checking Containment

Use the isBoundingBoxContained utility (internal to layout.ts) logic to check if text is within a rectangle: 
// This is done automatically by Kokokor
// Text bbox is considered contained if it's within the rectangle + tolerance
const isContained = isBoundingBoxContained(
  textBbox,
  rectangleBbox,
  5 // tolerance in pixels
);

​
Filtering Horizontal Lines

Sometimes horizontal lines appear inside rectangles (like underlined headings). Kokokor provides filterHorizontalLinesOutsideRectangles to remove these: 
import { filterHorizontalLinesOutsideRectangles } from 'kokokor';

const rectangles = [
  { x: 100, y: 100, width: 600, height: 80 }, // Heading box
];

const allHorizontalLines = [
  { x: 120, y: 170, width: 560, height: 2 }, // Line inside heading (underline)
  { x: 100, y: 2800, width: 600, height: 2 }, // Footer separator
];

// Filter out lines inside rectangles
const relevantLines = filterHorizontalLinesOutsideRectangles(
  rectangles,
  allHorizontalLines,
  5 // pixel tolerance
);

// Result: Only the footer separator remains
console.log(relevantLines.length); // 1
​
rectangles
BoundingBox[]
required
Array of rectangles to check against
​
horizontalLines
BoundingBox[]
required
Array of horizontal lines to filter
​
tolerance
number
default:5
Pixel tolerance for boundary checking. Extends rectangle boundaries to catch lines that are slightly outside due to OCR inaccuracies.
​
filtered
BoundingBox[]
Array of horizontal lines that are NOT contained within any rectangle

​
Complete Example

Here’s a complete example using both layout elements: 
import {
  reconstructParagraphs,
  filterHorizontalLinesOutsideRectangles,
  calculateDPI
} from 'kokokor';

// Layout elements detected from document
const allRectangles = [
  { x: 200, y: 50, width: 400, height: 60 },   // Title box
  { x: 150, y: 500, width: 500, height: 40 },  // Section heading
];

const allHorizontalLines = [
  { x: 220, y: 105, width: 360, height: 2 },   // Underline in title
  { x: 100, y: 2800, width: 600, height: 2 },  // Footer separator
];

// Filter out horizontal lines that are inside rectangles
const relevantHorizontalLines = filterHorizontalLinesOutsideRectangles(
  allRectangles,
  allHorizontalLines,
  5 // tolerance
);

console.log(`Found ${relevantHorizontalLines.length} relevant horizontal lines`);

// Calculate DPI if you have image and PDF dimensions
const dpi = calculateDPI(
  { width: 2480, height: 3508 }, // Image size
  { width: 595, height: 842 }     // PDF size (A4)
);

// Reconstruct paragraphs with layout elements
const result = reconstructParagraphs(
  {
    observations: ocrObservations,
    page: {
      width: 2480,
      height: 3508,
      dpiX: dpi.x,
      dpiY: dpi.y,
    },
    layout: {
      rectangles: allRectangles,
      horizontalLines: relevantHorizontalLines,
    },
  }
);

// Analyze results
console.log('\nDocument Structure:');
result.paragraphs.forEach((para, i) => {
  if (para.isHeading) {
    console.log(`\n=== ${para.text} ===`);
  } else if (para.isFootnote) {
    console.log(`  [Footnote] ${para.text}`);
  } else {
    console.log(`  ${para.text}`);
  }
});

​
Working with Detection Libraries

Many OCR and layout detection libraries can provide rectangle and line information:

​
From Surya Layout Detection

import { mapMatrixToBoundingBox } from 'kokokor';

// Surya layout results
const suryaLayout = {
  bboxes: [
    { bbox: [200, 50, 600, 110], label: 'heading' },
    { bbox: [100, 2800, 700, 2802], label: 'line' },
  ],
};

// Convert to Kokokor format
const rectangles = suryaLayout.bboxes
  .filter(item => item.label === 'heading')
  .map(item => mapMatrixToBoundingBox(item.bbox as [number, number, number, number]));

const horizontalLines = suryaLayout.bboxes
  .filter(item => item.label === 'line')
  .map(item => mapMatrixToBoundingBox(item.bbox as [number, number, number, number]));

​
From Custom Detection

// Your custom layout detection function
function detectLayoutElements(imagePath: string) {
  // ... your detection logic ...
  return {
    rectangles: [
      // Detected boxes/frames
      { x: 200, y: 50, width: 400, height: 60 },
    ],
    horizontalLines: [
      // Detected horizontal separators
      { x: 100, y: 2800, width: 600, height: 2 },
    ],
  };
}

const layout = detectLayoutElements('./document.png');

const result = reconstructParagraphs({
  observations: ocrObservations,
  page: pageContext,
  layout,
});

​
Tolerance Settings

The tolerance parameter controls how strictly containment is checked: 
// Tolerance = 0: Exact containment required
const lines = filterHorizontalLinesOutsideRectangles(
  rectangles,
  horizontalLines,
  0 // no tolerance
);
Use tolerance values between 5-10 pixels to account for OCR and layout detection inaccuracies. This helps catch lines and text that are slightly misaligned due to scanning artifacts.

​
Common Patterns

​
Academic Papers

// Academic papers often have:
// - Title in a box at the top
// - Horizontal line before references/footnotes

const academicLayout = {
  rectangles: [
    { x: 200, y: 100, width: 400, height: 80 }, // Title
  ],
  horizontalLines: [
    { x: 100, y: 2600, width: 600, height: 1 }, // References separator
  ],
};

​
Books and Reports

// Books may have:
// - Chapter headings in boxes
// - Multiple horizontal lines for sections

const bookLayout = {
  rectangles: [
    { x: 150, y: 200, width: 500, height: 50 },  // Chapter title
    { x: 150, y: 1500, width: 500, height: 40 }, // Section heading
  ],
  horizontalLines: [
    { x: 100, y: 3000, width: 600, height: 2 },  // Footer line
  ],
};

​
Next Steps

Advanced Configuration

Explore all configuration options

Surya Integration

Convert Surya layout detection results

Build docs developers (and LLMs) love

Get started for freeTalk to us