Performance and scalability
More than 14× the throughput of general-purpose servers, on the same CPU.
IQ-Stream was benchmarked against three of the best-known real-time servers on the exact traffic an AI product generates.
85,000/s
Events on one CPU
124,000/s
Events on two CPUs
2.4ms
p99 latency at 300 runs
0
Events lost on reconnect
The test
The exact traffic an AI product generates.
Simulated agent runs, each made of 429 AG-UI events over about 10 seconds. Each run has a streamed text answer, a tool call with streamed arguments, a 1.5-second pause while the tool runs, an 8 KB tool result, and a second answer, with runs starting and ending all the time and one browser in ten losing its connection mid-answer.
Each server ran alone, on the same CPU and memory, in its fastest configuration.
- Streamed text answers
- Tool call, streamed arguments
- 1.5 s pause while the tool runs
- 8 KB tool result
One CPU
IQ-Stream vs Mercure, Laravel Reverb, Soketi
| Concurrent runs | Server | Events delivered | CPU per event | p99 latency | Lost on reconnect |
|---|---|---|---|---|---|
| 100 | IQ-Stream | 100% | 57.5 µs | 0.97 ms | 0 |
| Mercure | 100% | 203.4 µs | 658 ms | 0.94% | |
| Mercure with history | 16% | 527.6 µs | 20.63 s | – | |
| Laravel Reverb | 55% | 424.8 µs | 7.97 s | – | |
| Soketi | 98% | 198.4 µs | 921 ms | 1.3% | |
| 300 | IQ-Stream | 100% | 49.0 µs | 2.4 ms | 0 |
| Mercure | 34% | 224.6 µs | 14.73 s | – | |
| Mercure with history | 3.9% | 868.1 µs | 26.68 s | – | |
| Laravel Reverb | 20% | 396.8 µs | 20.22 s | – | |
| Soketi | 42% | 182.8 µs | 11.62 s | – | |
| 1,000 | IQ-Stream | 98.9% | 23.5 µs | 168 ms | 0 |
| Mercure | 14% | 167.1 µs | 22.77 s | – | |
| Mercure with history | 1.5% | 603.9 µs | 27.22 s | – | |
| Laravel Reverb | 5.4% | 428.8 µs | 24.65 s | – | |
| Soketi | 12% | 199.9 µs | 22.33 s | – | |
| 2,000 | IQ-Stream | 99.5% | 11.4 µs | 92.6 ms | 0 |
| The other servers were not tested at this load: they had already fallen behind at 1,000 concurrent runs or fewer. | |||||
“Lost on reconnect” is left out (–) for servers that delivered under 90%: that far behind, the backlog hides it.
What the numbers say.
85,000/s
Events relayed on one CPU
Two thousand agent runs at once. At half that load, the best any general-purpose server reached was about 6,000, with its tokens arriving more than 20 seconds late.
2.4ms
p99 latency at 300 runs
None of the general-purpose servers keeps up with 300 concurrent runs. IQ-Stream delivers every event.
3.4×
Less CPU per event at 100 runs
Where two of them still keep up, IQ-Stream uses 3.4 to 3.5 times less CPU per event, with a p99 under one millisecond against 658 and 921 ms.
11µs
Per event at 2,000 runs
IQ-Stream gets cheaper per event as it gets busier, from 58 µs to 11 µs, because each turn of its event loop relays more events at once.
Scalability
From one CPU to every CPU of the machine.
IQ-Stream runs as a single process on one CPU, or as a cluster that uses every CPU of the machine: one worker per CPU, with a lightweight router in front. Each answer lives on one worker, so its tokens go straight from your backend to your users’ browsers, never crossing processes and never passing through a broker.
Two CPUs
Reverb and Soketi scaled out with Redis
| Concurrent runs | Server | Events delivered | CPU per event | p99 latency | Lost on reconnect |
|---|---|---|---|---|---|
| 200 | IQ-Stream cluster | 100% | 59.1 µs | 1.1 ms | 0 |
| Mercure | 96% | 228.8 µs | 851 ms | 0.90% | |
| Mercure with history | 7.3% | 830.1 µs | 24.65 s | – | |
| Laravel Reverb, scaled out with Redis | 20% | 1,136.2 µs | 19.44 s | – | |
| Soketi, scaled out with Redis | 52% | 445.6 µs | 8.46 s | – | |
| 600 | IQ-Stream cluster | 100% | 47.5 µs | 2.7 ms | 0 |
| Mercure | 32% | 239.4 µs | 16.59 s | – | |
| Mercure with history | 3.0% | 623.7 µs | 26.68 s | – | |
| Laravel Reverb, scaled out with Redis | 8.6% | 910.5 µs | 24.17 s | – | |
| Soketi, scaled out with Redis | 12% | 635.9 µs | 22.33 s | – | |
| 4,000 | IQ-Stream cluster | 75% | 15.5 µs | 2.34 s | – |
| The other servers were not tested at this load: they had already fallen behind at 600 concurrent runs or fewer. | |||||
IQ-Stream scales with its CPUs.
On two CPUs it delivered about 124,000 events per second, and at every load it kept up with, its cost per event stayed the same as on one.
General-purpose servers do not.
Mercure reached about 8,300 events per second on two CPUs. Soketi and Reverb, scaled out the way their documentation describes, delivered fewer events than on one CPU, at two to four times the CPU per event, because every event has to cross Redis to reach the right process.
On the same two CPUs, IQ-Stream moved
9×
what Mercure did
17×
what Soketi did
35×
what Reverb did
Memory
Light on memory.
A whole agent run, kept in full so any browser can resume it, costs IQ-Stream about 80–95 KiB. A 2 GB container holds about 1,400 complete 4,000-token answers in flight at once. IQ-Stream sizes itself from its container’s memory limit and refuses new work before it runs out, so a traffic spike never crashes the process and the answers already running.
80–95KiB
Per agent run, kept in full
1,400
4,000-token answers in a 2 GB container
Specialization vs general purpose
Every AI product streams. The streaming layer was built for something else.
Chat assistants, copilots, agents and AI search all send their answers word by word. But WebSocket hubs, pub/sub brokers and notification servers were designed to deliver a chat message or a notification now and then.
An LLM answer is not like that. It is hundreds of small events, a few dozen milliseconds apart, for ten seconds or more, with pauses while a tool runs and the occasional large payload, for every user at once.
Put that traffic through a general-purpose server and three things happen:
01 · CPU
It runs out of CPU long before you expect.
Every token pays the full price of a message built for a different job.
02 · Latency
Latency piles up.
Once the server falls behind, tokens arrive in bursts, seconds late, and the “typing” effect your users expect falls apart.
03 · Reconnects
Dropped connections lose tokens.
The browser reconnects and the middle of the answer is gone. The user sees a broken answer.
Doing one thing is what makes it fast.
IQ-Stream was built to remove all three. It keeps every running answer in memory, for as long as it is being written and no longer, so replay costs nothing on the hot path. Just the shortest possible path from your model to your user’s screen.
Nothing else
- topics
- channels
- presence
- message history
- broker
- database
Benchmark
How we measured.
The figures on this page come from IQ-Stream’s reproducible benchmark suite, which runs with Docker alone. It streams simulated AG-UI agent runs through each server, one at a time, on the same CPU and memory limits, and measures the events delivered, the CPU and memory used, the latency each event saw from the moment the model produced it, and the events browsers lost when their connection dropped.
Each competitor ran in its fastest documented setup: Mercure with its in-memory transport, and again with its default transport that keeps history; Laravel Reverb with its recommended event loop and production settings; Soketi with its official image; Reverb and Soketi scaled out with Redis on two CPUs, as their documentation describes. Every server checked that each browser was allowed to read the stream it asked for.
The results were measured on a single machine (Intel Core Ultra 7 155H). Absolute numbers depend on the hardware: compare servers within one run, on one machine.
See the numbers for your environment.
Ask us to run the benchmark on your own infrastructure.