Skip to main content

Documentation Index

Fetch the complete documentation index at: https://mintlify.com/phpaisdk/groq/llms.txt

Use this file to discover all available pages before exploring further.

Streaming lets your application display tokens as the model produces them rather than waiting for the entire response to be assembled. This makes long-form responses feel substantially faster to end users because the first words appear almost immediately after the request is sent.

​
How Streaming Works

When you call ->stream() instead of ->run(), the SDK opens a server-sent events connection to Groq’s API. The model writes tokens into that stream as it generates them. The SDK surfaces those tokens to your code through a chunks() generator so you can echo, buffer, or process each piece independently. After the stream is exhausted you call ->run() on the stream object to get the same TextModelResponse you would receive from a non-streaming call — with text, usage, and providerMetadata all populated.

​
Basic Streaming Example

Replace ->run() with ->stream(), then iterate the chunks. Call ->run() on the stream object when you need token counts or provider metadata.
stream-basic.php
use AiSdk\Generate;
use AiSdk\Groq;

$stream = Generate::text('Tell me a short story about a lighthouse keeper.')
    ->model(Groq::model('llama-3.3-70b-versatile'))
    ->stream();

foreach ($stream->chunks() as $chunk) {
    echo $chunk;
}

// Collect the final response after the stream completes.
$result = $stream->run();

echo PHP_EOL . 'Input tokens:  ' . $result->usage->inputTokens;
echo PHP_EOL . 'Output tokens: ' . $result->usage->outputTokens;
->run() must be called after the foreach loop finishes. Calling it before the stream is exhausted will not yield correct usage counts.

​
Streaming in a CLI Application

The following is a complete command-line script that streams a response and prints a usage summary once the stream finishes.
cli-stream.php
#!/usr/bin/env php
<?php

declare(strict_types=1);

require __DIR__ . '/vendor/autoload.php';

use AiSdk\Generate;
use AiSdk\Groq;

$stream = Generate::text()
    ->model(Groq::model('llama-3.3-70b-versatile'))
    ->instructions('You are a helpful assistant. Be thorough but concise.')
    ->prompt('Explain how PHP generators work, with a practical example.')
    ->stream();

foreach ($stream->chunks() as $chunk) {
    echo $chunk;
}

$result = $stream->run();

echo PHP_EOL . PHP_EOL;
echo '--- Usage ---' . PHP_EOL;
echo 'Input tokens:  ' . $result->usage->inputTokens  . PHP_EOL;
echo 'Output tokens: ' . $result->usage->outputTokens . PHP_EOL;
echo 'Finish reason: ' . $result->providerMetadata['groq']['choice_finish_reason'] . PHP_EOL;
Run it directly from the terminal:
Terminal
php cli-stream.php

​
Streaming in a Web Context

Browsers buffer output by default. You need to flush PHP’s output buffers on every chunk to ensure tokens reach the client as they are produced.
1

Disable output buffering and set the content type

Send the correct headers before any output so the browser treats the response as a stream.
web-stream.php
<?php

declare(strict_types=1);

require __DIR__ . '/vendor/autoload.php';

use AiSdk\Generate;
use AiSdk\Groq;

// Disable any output buffering layers that may be active.
while (ob_get_level() > 0) {
    ob_end_flush();
}

header('Content-Type: text/plain; charset=utf-8');
header('X-Accel-Buffering: no'); // Disable Nginx proxy buffering.
2

Stream the response and flush on each chunk

web-stream.php (continued)
$stream = Generate::text('Summarise the latest trends in large language model research.')
    ->model(Groq::model('llama-3.3-70b-versatile'))
    ->stream();

foreach ($stream->chunks() as $chunk) {
    echo $chunk;
    ob_flush();
    flush();
}

$result = $stream->run();
If you are building a chat interface, consider using the text/event-stream content type and formatting each chunk as a Server-Sent Event (data: ...\n\n). This lets your JavaScript client parse chunks reliably via the EventSource API.
Some reverse proxies (nginx, Cloudflare, AWS ALB) buffer responses regardless of PHP-level flushing. Set proxy_buffering off in nginx or enable streaming pass-through in your load balancer configuration to ensure chunks reach the browser without delay.

​
Checking Streaming Support

All Groq models registered with the package declare Streaming as a Native capability. You can verify this at runtime with ->supports().
check-streaming.php
use AiSdk\Capability;
use AiSdk\Groq;

$model = Groq::model('llama-3.3-70b-versatile');

if ($model->supports(Capability::Streaming)) {
    echo 'Streaming is supported.';
}
To check a model you registered yourself, the same call works because capability resolution follows the same path through the SDK’s model registry.
check-custom-model.php
use AiSdk\Capability;
use AiSdk\Groq;

Groq::registerModel('llama-5-70b', capabilities: [
    Capability::TextGeneration,
    Capability::Streaming,
    Capability::ToolCalling,
]);

$supported = Groq::model('llama-5-70b')->supports(Capability::Streaming); // true
The table below shows the streaming status for every built-in model.
Model IDStreaming
llama-3.1-8b-instant✅ Native
llama-3.3-70b-versatile✅ Native
meta-llama/llama-4-scout*✅ Native
meta-llama/llama-4-maverick*✅ Native
moonshotai/kimi*✅ Native
openai/gpt-oss-20b✅ Native
openai/gpt-oss-120b✅ Native

​
Full API Reference for Streaming

MethodReturnsDescription
->stream()Stream objectOpens the SSE connection and returns a lazy stream
$stream->chunks()Generator<string>Yields each text chunk as a string as it arrives
$stream->run()TextModelResponseBlocks until the stream is exhausted; returns the full response
The TextModelResponse returned by $stream->run() is identical to the one returned by ->run() in non-streaming mode and carries text, usage->inputTokens, usage->outputTokens, and providerMetadata['groq'].

Build docs developers (and LLMs) love

Get started for freeTalk to us