For cloud and infrastructure providers
Your customers are building AI products, and every one of them needs to stream tokens to the browser.
Today they assemble it from WebSocket servers, Redis and custom reconnection code, or pay a general-purpose real-time service per message for traffic it was never designed for.
IQ-Stream lets you offer LLM streaming as a first-class managed service: a dedicated endpoint your customers point their backends and browsers at, with replay included and no infrastructure for them to run.
≈ 2,000
Concurrent agent runs on one CPU
< 300
Where general-purpose servers fall behind
0
Brokers, databases or disks in the data path
Density is margin
A service like this is priced per customer and run per CPU. IQ-Stream serves around 2,000 concurrent agent runs on a single CPU, where general-purpose servers fall behind before 300, and the margin per customer grows accordingly.
Take an AI product with 5,000 agent runs streaming at the same time, each like the ones in our benchmark: about 43 events per second, roughly 214,000 events per second in total.
| IQ-Stream | Fastest general-purpose server tested | |
|---|---|---|
| CPU per event (measured) | 49 µs | 203 µs |
| CPUs busy relaying | ≈ 10.5 | ≈ 43.5 |
| vCPUs provisioned at 70% utilization | ≈ 15 | ≈ 62 |
| Illustrative cost at $30 per vCPU per month | ≈ $450/ month | ≈ $1,860/ month |
| Replay on reconnect | Included | Not included |
| Redis, database or disk needed | None | None without replay. Disk for every event with it |
| p99 latency at the measured load | 2.4 ms | 658 ms |
The comparison is generous to the general-purpose server: it uses its best figure, measured at a load where it still kept up, and assumes it scales linearly with CPUs, which the benchmark shows it does not. Mercure gained only 1.4× from a second CPU, and Soketi and Reverb lost throughput when scaled out. Adding replay to it makes things much worse: with history on, it costs 2.6 times more CPU per event and could not keep up even with 100 concurrent runs.
And it keeps scaling: on the same two CPUs, IQ-Stream moved
9×
what Mercure did
17×
what Soketi did
35×
what Reverb did
Where the savings come from
3.4 to 9 times less CPU per event than general-purpose servers, and up to 35 times more events on the same two CPUs.
No Redis cluster to relay events between nodes, no database for history, no disk to provision, monitor and back up.
Reconnection, ordering, end-of-answer signals and slow-client handling are done.
Tokens reach the screen in milliseconds, not seconds, even at peak.
Why IQ-Stream fits a provider’s economics
Per-customer credentials, per-customer limits and per-customer counters come with the server. Plans map directly onto limits, all on the same fleet, changed live from your billing database.
50
Starter app · concurrent answers
5,000
Enterprise app · concurrent answers
IQ-Stream looks up customers in the store you already run (PostgreSQL, MySQL, Redis or an internal API) and caches what it reads. No separate customer database to sync, no extra service to operate.
Each customer’s live counters (active answers, connected viewers, memory in use) are available per app, ready to feed dashboards, alerts and usage-based billing.
One customer’s spike cannot take capacity from another. A customer at its limit gets a clear “limit reached” answer for its own new streams, while every other customer keeps streaming.
No broker, no database, no disk, no cross-node message bus in the data path. A server that keeps nothing on disk upgrades with a rolling restart, and grows with the CPUs you give it.
Signed images, SBOMs, provenance records, non-root execution and air-gapped installation fit the compliance reviews of providers serving finance, healthcare and the public sector. Answer content is never logged or written to disk.
Many isolated applications
IQ-Stream can host the streams of many independent applications, belonging to different teams or different customers, on the same servers, safely.
Each app signs its own tokens with its own secret. One app can never publish to, or read, another’s streams.
Cap each app’s concurrent answers and memory. An app that hits its limit loses only its own new streams. Every other app keeps streaming as if nothing happened.
Concurrent answers, memory, answer size, maximum duration, viewers per answer, timeouts. Change them in your own database and they apply within a minute, without a restart.
A JSON file, PostgreSQL, MySQL or Redis, or your own internal service through a small connector. A slow or failing store never takes streaming down: IQ-Stream caches answers and keeps serving known apps through a store outage.
Each app gets a monitoring key that reads only its own live counters (active answers, viewers, memory) and can never publish or read a stream.
Fake credentials are rejected before they cost a lookup, and floods of unknown app keys are rate-limited, so a public endpoint stays standing.
Single-tenant
Run IQ-Stream next to your application and give it your app’s credentials. Your backend signs a short-lived token for each answer: one to publish, one to watch. The browser gets only the token to watch, and only for that answer.
Dedicated instance
The same image runs as a single-tenant instance in their account or private cloud, with the same behaviour and the same guarantees.
Operate it
IQ-Stream ships as a ready-to-use Docker image for amd64 and arm64. One command starts it. It needs no database, no Redis, no disk and no message broker: everything it does happens in memory.
Signed releases, with a software bill of materials (SBOM) and a build provenance record for every platform.
Runs as a non-root user, with read-only application files.
Graceful shutdown: on a rolling deploy, running answers get time to finish before the process exits.
Health checks for load balancers and orchestrators out of the box.
Structured JSON logs that never contain tokens, secrets or the content of your customers’ answers.
Mirrorable and air-gap friendly: copy the image, with its signatures, into a private registry or carry it into an isolated network.
One process or every CPU: a single setting switches between the two modes.
FAQ
Yes. Multi-tenancy is built in: separate credentials, separate limits and separate counters for every app, with no customer able to read, write or crowd out another’s streams.
Yes. IQ-Stream is a single Docker image for amd64 and arm64, signed and verifiable, that runs anywhere Docker runs, air-gapped networks included.
No. IQ-Stream keeps an answer in memory only while it is being generated, so a browser that reconnects can resume. Once the answer ends and the browsers watching it have read it, IQ-Stream drops it. It never writes content to disk and never logs it.
Answers in flight on that process are lost, and browsers are told so explicitly, so the app can load the result from its backend. Planned restarts drain gracefully: running answers get time to finish first.
The streaming layer AI products should have had from the start: built for tokens, fast on a single CPU, resilient to every dropped connection, and ready to host one app or thousands.