decodeStreamSync

Overview

decodeStreamSync converts TOON-formatted lines into a stream of structured JSON events without building the complete value tree in memory. This enables memory-efficient parsing, custom transformations, and progressive processing of TOON data.

This is the synchronous variant that accepts and returns synchronous iterables. For async sources like file streams or network responses, use decodeStream instead.

Function Signature

function decodeStreamSync(
  lines: Iterable<string>,
  options?: DecodeStreamOptions
): Iterable<JsonStreamEvent>

Parameters

Iterable<string>

required

An iterable of TOON lines (without trailing newlines). Can be an array of strings, a generator, or any synchronous iterable.

DecodeStreamOptions

Optional configuration object for decoding behavior.

number

default:2

Number of spaces per indentation level. Must match the indentation used in the TOON input.

boolean

default:true

When true, enforces strict validation of array lengths, tabular row counts, and blank line restrictions.

Path expansion not supported: The expandPaths option from decode() is not available in streaming mode. Streaming decoders emit raw events without post-processing transformations.

Return Value

Iterable<JsonStreamEvent>

A synchronous iterable that yields JSON stream events. Each event represents a structural element of the JSON data model.

Event Types

The decoder emits six types of events:

{ type: "startObject" }

Marks the beginning of a JSON object.

{ type: "endObject" }

Marks the end of a JSON object.

{ type: "startArray", length: number }

Marks the beginning of a JSON array. The length field indicates the declared array length from the TOON header.

{ type: "endArray" }

Marks the end of a JSON array.

{ type: "key", key: string, wasQuoted?: boolean }

Represents an object key. The optional wasQuoted field is true when the key was quoted in the TOON source.

{ type: "primitive", value: JsonPrimitive }

Represents a primitive value (string, number, boolean, or null).

Usage Examples

Basic Event Streaming

Process TOON lines event-by-event for custom handling:

import { decodeStreamSync } from '@toon-format/toon'

const lines = [
  'name: Alice',
  'age: 30',
  'active: true'
]

for (const event of decodeStreamSync(lines)) {
  console.log(event)
}

// Output:
// { type: 'startObject' }
// { type: 'key', key: 'name' }
// { type: 'primitive', value: 'Alice' }
// { type: 'key', key: 'age' }
// { type: 'primitive', value: 30 }
// { type: 'key', key: 'active' }
// { type: 'primitive', value: true }
// { type: 'endObject' }

Array Processing

Handle array events with declared lengths:

import { decodeStreamSync } from '@toon-format/toon'

const lines = [
  'users[2]:',
  '  - name: Alice',
  '  - name: Bob'
]

for (const event of decodeStreamSync(lines)) {
  if (event.type === 'startArray') {
    console.log(`Processing ${event.length} items...`)
  }
  console.log(event)
}

// Output includes:
// { type: 'startArray', length: 2 }

Custom Value Builder

Build values with custom logic by handling events manually:

import { decodeStreamSync } from '@toon-format/toon'

function buildCustomValue(lines: string[]) {
  const events = decodeStreamSync(lines)
  const stack: any[] = []
  let current: any = null

  for (const event of events) {
    switch (event.type) {
      case 'startObject':
        const obj = {}
        if (current !== null) stack.push(current)
        current = obj
        break

      case 'key':
        stack.push(event.key)
        break

      case 'primitive':
        const key = stack.pop()
        current[key] = event.value
        break

      case 'endObject':
        if (stack.length > 0) {
          const parent = stack.pop()
          const parentKey = stack.pop()
          parent[parentKey] = current
          current = parent
        }
        break
    }
  }

  return current
}

const lines = ['name: Alice', 'score: 95']
console.log(buildCustomValue(lines))
// { name: 'Alice', score: 95 }

Filtering Values During Decode

Use streaming events to filter or transform data on-the-fly:

import { decodeStreamSync } from '@toon-format/toon'

function* filterSensitiveKeys(lines: string[]) {
  let skipNext = false

  for (const event of decodeStreamSync(lines)) {
    if (event.type === 'key' && event.key === 'password') {
      skipNext = true
      continue // Skip the key event
    }

    if (skipNext && event.type === 'primitive') {
      skipNext = false
      continue // Skip the value event
    }

    yield event
  }
}

const lines = [
  'username: alice',
  'password: secret123',
  'email: alice@example.com'
]

for (const event of filterSensitiveKeys(lines)) {
  console.log(event)
}
// password key and value are omitted

Reading from a Generator

Work with lazy line generators for memory efficiency:

import { decodeStreamSync } from '@toon-format/toon'

function* readLinesFromFile(filePath: string) {
  // Simplified example - actual implementation would read line-by-line
  const content = readFileSync(filePath, 'utf-8')
  yield* content.split('\n')
}

const lineGenerator = readLinesFromFile('data.toon')
const events = decodeStreamSync(lineGenerator)

for (const event of events) {
  // Process events as they're decoded
  if (event.type === 'primitive') {
    console.log('Value:', event.value)
  }
}

Custom Indentation

Decode TOON with non-standard indentation:

import { decodeStreamSync } from '@toon-format/toon'

const lines = [
  'user:',
  '    name: Alice',      // 4-space indent
  '    profile:',
  '        bio: Engineer' // 8-space indent
]

const events = decodeStreamSync(lines, { indent: 4 })

for (const event of events) {
  console.log(event)
}

Strict vs. Non-Strict Mode

Control validation behavior with the strict option:

import { decodeStreamSync } from '@toon-format/toon'

// Strict mode (default): throws on mismatched counts
const strictLines = [
  'items[3]:', // Declares 3 items
  '  - apple',
  '  - banana'  // Only 2 items provided
]

try {
  for (const event of decodeStreamSync(strictLines)) {
    // Will throw: expected 3 items, got 2
  }
} catch (error) {
  console.error(error.message)
}

// Non-strict mode: allows mismatched counts
const events = decodeStreamSync(strictLines, { strict: false })
for (const event of events) {
  console.log(event) // Processes without error
}

Event Stream Lifecycle

Understanding the event sequence for different TOON structures:

const lines = ['key: value']
// Events:
// startObject → key → primitive → endObject

const lines = ['items[2]:', '  - a', '  - b']
// Events:
// startArray → primitive → primitive → endArray

const lines = ['user:', '  name: Alice']
// Events:
// startObject → key → startObject → key → primitive → endObject → endObject

Performance Considerations

Memory Efficient

Only the current event is in memory at any time, enabling processing of arbitrarily large TOON files.

Early Termination

Stop iteration at any point to avoid processing unnecessary data.

No Buffering

Events are yielded as soon as they’re parsed with zero buffering overhead.

Generator-Friendly

Works seamlessly with generator functions and lazy iterables.

Error Handling

Errors are thrown synchronously when invalid TOON syntax is encountered:

import { decodeStreamSync } from '@toon-format/toon'

const invalidLines = [
  'key: value',
  '  invalid indentation' // Unexpected indent
]

try {
  for (const event of decodeStreamSync(invalidLines)) {
    console.log(event)
  }
} catch (error) {
  console.error('Decode error:', error.message)
  // Handle syntax errors
}

decodeStream

Async variant for streaming sources like file readers and network responses

decodeFromLines

Decode lines directly to a value (supports path expansion)

decode

Standard decoder that builds the complete value tree

Event Builder Guide

Learn how to build values from event streams

Type Definitions

type JsonStreamEvent
  = | { type: 'startObject' }
    | { type: 'endObject' }
    | { type: 'startArray', length: number }
    | { type: 'endArray' }
    | { type: 'key', key: string, wasQuoted?: boolean }
    | { type: 'primitive', value: JsonPrimitive }

type JsonPrimitive = string | number | boolean | null

interface DecodeStreamOptions {
  indent?: number      // Default: 2
  strict?: boolean     // Default: true
  expandPaths?: never  // Not supported in streaming mode
}

Core Functions

Streaming

Configuration

decodeStreamSync

Overview

Function Signature

Parameters

Return Value

Event Types

Usage Examples

Basic Event Streaming

Array Processing

Custom Value Builder

Filtering Values During Decode

Reading from a Generator

Custom Indentation

Strict vs. Non-Strict Mode

Event Stream Lifecycle

Performance Considerations

Memory Efficient

Early Termination

No Buffering

Generator-Friendly

Error Handling

decodeStream

decodeFromLines

decode

Event Builder Guide

Type Definitions

​Overview

​Function Signature

​Parameters

​Return Value

​Event Types

​Usage Examples

​Basic Event Streaming

​Array Processing

​Custom Value Builder

​Filtering Values During Decode

​Reading from a Generator

​Custom Indentation

​Strict vs. Non-Strict Mode

​Event Stream Lifecycle

​Performance Considerations

Memory Efficient

Early Termination

No Buffering

Generator-Friendly

​Error Handling

​Related Functions

decodeStream

decodeFromLines

decode

Event Builder Guide

​Type Definitions

Overview

Function Signature

Parameters

Return Value

Event Types

Usage Examples

Basic Event Streaming

Array Processing

Custom Value Builder

Filtering Values During Decode

Reading from a Generator

Custom Indentation

Strict vs. Non-Strict Mode

Event Stream Lifecycle

Performance Considerations

Error Handling

Related Functions

Type Definitions