How an answer types itself out word by word: streaming over HTTP

You ask an AI assistant a question, and the answer doesn’t appear all at once - it comes word by word, as if someone were typing it live. That’s not a visual effect bolted on for show. That’s how streaming works: the server’s response doesn’t arrive as one lump but in pieces, and the page shows each piece the moment it lands.

And there’s nothing AI-specific about it - it’s a plain HTTP mechanism, older than LLMs. They just made it an everyday thing, but underneath it’s the same web you already know.

#The answer doesn’t have to arrive all at once

By default we think of a request like this: you send it, you wait, and you get the whole answer in one piece. First the server prepares it in full, then sends it, and the browser shows it once the last byte arrives.

But it doesn’t have to be. An HTTP response body is a stream - a sequence of bytes that can flow in gradually. The server doesn’t have to wait until it has the whole thing: it can send it in chunks as it produces them, keeping the connection open until it’s done. The browser receives those chunks as they come.

For an AI answer this fits perfectly: the model generates text piece by piece anyway, so instead of waiting for the whole paragraph, the server sends each fragment right away. The user sees the first words after a fraction of a second, not after several seconds of silence. The total time may be the same - but the perceived time is much shorter, because something happens immediately.

Switch the mode and play it once as “all at once”, once as “streaming” - the difference shows from the very first word:

#The simplest way: Server-Sent Events

If you only need one direction - the server sends, the browser receives - there’s a ready-made, simple tool: Server-Sent Events (SSE). In the browser all it takes is new EventSource(url) and a listener on the message event - and that’s basically it. EventSource opens the connection itself and receives each chunk, and when the connection drops, it tries to reconnect on its own. So once the answer is done you have to close the connection yourself (source.close()) - otherwise EventSource will just reconnect. On its side the server marks the response with the Content-Type: text/event-stream header and sends each chunk as lines starting with data:, separated by a blank line.

Simple - but you pay for that simplicity with limits. SSE is one-way (server → browser only), works only with the GET method, and won’t let you add your own headers to the request. For a plain “the server talks, the client listens” that’s no problem. It gets worse when you want to send the question with POST or add a header with a token - then SSE gets in your way.

#With full control: fetch and a stream

This is where the second way comes in, more flexible because it works one level lower. A plain fetch gives you a stream too: response.body is a ReadableStream, a sequence of chunks you can read one by one as they arrive.

const response = await fetch("/api/answer", {
  method: "POST",
  body: JSON.stringify({ question }),
});

const reader = response.body.getReader();
const decoder = new TextDecoder();

while (true) {
  const { value, done } = await reader.read();
  if (done) break;
  appendToUI(decoder.decode(value, { stream: true }));
}

reader.read() returns the next chunk as soon as it lands, or done: true when the stream ends. One thing that’s easy to miss: value isn’t text but raw bytes (Uint8Array). That’s why we turn them into text with a TextDecoder, and this is where the { stream: true } flag matters. Chunks are split wherever the transmission happened to break, not on character boundaries - and some characters take more than one byte (the letter “é” is two bytes, an emoji up to four). So a character can land split in half: the first byte at the end of one chunk, the rest at the start of the next. Without the flag the decoder would put an unreadable replacement character (�) there; with it, it sets the incomplete character aside and assembles it only once the rest arrives.

In return for these few lines you get full control: POST, your own headers, any data format. And this is exactly the path the LLM libraries take under the hood - they send the question with POST and read the response stream exactly like this.

#Which one when

• SSE - when “the server sends, the client listens” is enough and you want it in five minutes: notifications, a progress bar, a simple feed. Auto-reconnect for free.
• fetch with a stream - when you need POST, your own headers, or an unusual format (which usually means AI). A bit more code, a lot more freedom.
• WebSocket - when traffic needs to flow both ways at once (chat, a game, collaborative editing). It’s a separate tool; for just streaming a response from the server it’s overkill.

#The catches

Three things break streaming most often:

1. Buffering along the way. Streaming works only when every layer between the server and the browser passes chunks through right away. If a proxy or CDN buffers the response to hand it back whole, the whole effect is gone - the user waits for the end again. That’s why streamed endpoints are served so that nothing along the way buffers them (on nginx that’s X-Accel-Buffering: no) or caches them (Cache-Control: no-cache).
2. Cancellation. The user closed the tab or asked a new question halfway through the answer? The stream has to be stopped so you don’t keep reading it forever. That’s what AbortController is for - the same one you cancel a plain fetch with.
3. An error mid-way. With a plain request you get the error at the start, before you’ve shown anything. With a stream the connection can drop partway through, when part of the answer is already on screen. You have to handle that separately: show that the answer is incomplete, instead of pretending it succeeded.

Next time an answer types itself out word by word, you’ll know it’s not an interface trick. A response body is a stream of bytes, and the whole craft comes down to one thing: read them as they arrive, instead of waiting for the last one.