Why AI Answers Need to Be Streamed

This article is for founders, CMOs and marketing leaders who have added AI to their website, a chat, a search box or a tool, and heard the same complaint we did from a client: "it looks slow." Here is why it happens, and the three fixes that work today.

Why do AI answers on a website need to be streamed?

Because people expect an answer in under a second, and an AI model takes several seconds or more to write a full answer. On a typical website the question goes from the browser to your server to the AI and back, and if each step waits for the complete answer, the visitor sees a blank screen the whole time. Streaming sends each piece of the answer as soon as the AI writes it, so the first words appear in about a second. Combine it with a model tuned for speed and with using AI only for questions that need it.

AI Summary

People expect a website to answer in under a second. AI cannot do that today: a good model takes several seconds, sometimes much longer, to write a full answer. On a typical website the question travels from the browser to your server, from your server to the AI, and all the way back, and if each step waits for the complete answer, the visitor stares at a spinner the whole time. There are three fixes. Tune the model so it answers faster. Use AI only when a question actually needs it. And stream the answer, so the first words appear in about a second and the rest flows in as the AI writes it. Streaming does not make the AI faster, but it makes the wait feel like a conversation instead of a broken page. At the end of this article is a prompt you can give your AI coding tool to add streaming to your own site.

Users Want Sub-Second Answers

Years of fast search engines and apps have trained people to expect an answer the moment they press Enter. Usability research has said the same thing for decades: about a tenth of a second feels instant, about one second keeps a person's train of thought, and after about ten seconds people lose focus and start to wonder whether something is broken.

A blank screen with a spinner is the worst version of that wait. The visitor cannot tell whether the answer is coming in two seconds or never. Many of them leave, and the ones who stay already think less of the tool, and of your company, before they read a word.

Why AI Cannot Answer That Fast Today

An AI model does not look up an answer. It writes one, a small piece at a time, and each piece depends on the pieces before it. A short answer can take a few seconds to write; a long or carefully reasoned one can take ten, twenty or thirty seconds. Models that think before they answer add more time before the first word.

That is not a bug in your website, and no amount of server tuning makes a long answer appear in under a second. What you control is how long the visitor waits before something useful appears, and how much of the work needs AI at all.

The Typical Architecture: Website to Server to AI and Back

Almost every AI feature on a website works the same way:

1. Website

The visitor types a question in the browser

2. Your server

Adds your instructions, data and keys, then calls the AI

3. The AI model

Writes the answer one small piece at a time

Every question makes the round trip: website to server to AI, and back the same way. Without streaming, each step waits for the whole answer before passing it on.

The server in the middle matters. It keeps your AI provider's key secret, adds your instructions and your own content to the question, and records the conversation. But it is also one more place where the answer can get stuck. The AI may be sending the answer piece by piece while your server collects all of it before replying, or your hosting platform or a proxy may hold the response until it is complete. The visitor's browser then gets nothing until the very end.

This is why the same model can feel fast in ChatGPT or Claude and slow on your website. Those apps stream every answer. Many websites, built the usual way with one request and one complete response, do not.

Why It Looks Slow

Picture the same ten-second answer two ways. In the first, the visitor sees a spinner for ten seconds and then a wall of text. In the second, the first words appear after about a second and the answer keeps growing as they read. The total time is the same. The first feels broken; the second feels like someone is talking to you.

When our client said their AI "looks slow," the model was not the main problem. The page simply showed nothing until the whole answer had made the full round trip.

Fix	What you change	What the visitor notices
Tune the model	A smaller or faster model, a lower reasoning setting, shorter answers, and cached instructions	Makes the first words and the full answer arrive sooner
Use AI only when needed	Answer fixed questions, lookups and forms in normal code; save AI for questions that need judgment	Many answers become instant because they skip the AI entirely
Stream the answer	Pass each piece of the answer from the AI, through your server, to the page as soon as it is written	The visitor sees words in about a second instead of a blank screen

Fix 1: Tune the Model

Start by making the AI itself answer sooner:

Pick the right size of model. The largest model is rarely needed for a website chat or a product question. Smaller, faster models from the same provider often answer just as well for focused tasks, in a fraction of the time and cost.
Turn down the thinking. Many models have a reasoning, thinking or effort setting. A website answer usually does not need deep reasoning, and turning it down shortens the wait before the first word.
Ask for shorter answers. Tell the model to answer in a few sentences and offer more if the visitor wants it. Shorter answers finish sooner.
Cache what repeats. Your instructions and site content are the same on every question. Prompt caching, offered by the major AI providers, lets the model reuse them instead of reading them again each time, which cuts both delay and cost.

Fix 2: Use AI Only When Needed

The fastest AI answer is the one you do not have to generate. Look at the questions people actually ask:

Fixed answers such as your address, hours or pricing page can be shown instantly from your own content.
Lookups such as order status, a product's specs or whether you serve a region are database queries, not AI questions.
Simple choices such as "talk to sales" or "book a demo" work better as buttons and forms.

Save the AI for questions that need judgment: comparing options, explaining something in the visitor's own terms, or pulling together several pages of your site into one answer. Everything else gets faster, and your AI bill goes down.

Fix 3: Stream the Answer

What is streaming? Instead of waiting for the AI to finish the whole answer and then sending it in one piece, streaming sends each piece of the answer the moment it is written. Think of the difference between a person writing you a letter and a person talking to you. The letter arrives all at once, after a wait. Talking starts right away.

On a website, streaming has to work at every step of the round trip:

The AI call. Your server asks the AI provider for a streamed response. All of the major providers support this.
Your server. It passes each piece on to the browser as it arrives, instead of collecting the whole answer first.
Your hosting. The platform, any proxy, and any compression in between must not hold the response until it is complete. Most modern hosting supports streamed responses, but sometimes you must choose the right kind of function or setting.
The page. The browser reads the response as it arrives and adds the words to the screen.

If any one of these steps waits for the full answer, the visitor waits too. That is why "we turned on streaming" sometimes changes nothing: one step in the middle is still buffering.

How We Do It on This Website

The Ask BuildMarketing.ai chat on this site uses all three fixes. It runs on a model setting tuned for quick answers with a low reasoning effort, our site content is cached between questions, and questions about things like booking a call go to a form instead of the AI. Every answer is streamed: the browser sends the question to a Netlify Function, the function asks Claude for a streamed answer, and each piece of text goes straight through to the page. The conversation is saved after the last word, so record keeping never delays the first one. Try it with the chat button on any page and watch the answer appear.

A Prompt to Add Streaming to Your Website

If your website was built with an AI coding tool such as Claude Code, Cursor or GitHub Copilot, or by a developer who uses one, you can give it this prompt. It asks the tool to trace the full round trip first, then fix every step that waits for the whole answer, and to measure the result.

PROMPT: ADD STREAMING TO YOUR AI FEATURE

You are working in my website's codebase. Our AI feature feels slow: visitors type a question, see a spinner, and wait for the whole answer before anything appears. Make the answer stream onto the page word by word as the AI writes it.

Before you change anything:
1. Find the full path of one question: the page or component the visitor types into, the server endpoint or function it calls, and the AI API call (OpenAI, Anthropic, Gemini or another provider).
2. Tell me which step waits for the complete answer today. Check the AI call, the server response, the hosting platform and the browser code.

Then make these changes:
3. Call the AI provider's streaming API (for example stream: true, or the SDK's stream helper) and forward each piece of text as it arrives.
4. Return a streamed response from the server: a ReadableStream with plain text, or Server-Sent Events (text/event-stream). Do not collect the whole answer before sending it.
5. Make sure nothing between the server and the browser buffers the response: check the hosting platform's function settings, any proxy or CDN, compression and middleware. Use a function type that supports streaming responses.
6. In the browser, read the response with fetch() and response.body.getReader(), decode each chunk with TextDecoder, and append it to the answer on screen as it arrives. Keep the page scrolled to the newest text.
7. Show a small "thinking" indicator only until the first words arrive, then remove it.
8. Handle errors part-way through: keep the text already shown and add a short message with a way to try again. Let the visitor stop or cancel an answer.
9. Keep anything that needs the full answer (saving the chat, logging, lead alerts) running after the stream ends, without delaying the first words.

Also check for speed problems streaming will not fix:
10. List what is sent to the AI on every question (system prompt size, history, documents). Turn on prompt caching for the parts that repeat, if the provider supports it.
11. Tell me whether a smaller or faster model, or a lower reasoning or thinking setting, would give good enough answers for this feature.
12. Point out questions that do not need AI at all (fixed answers, lookups, forms) and could be answered instantly in normal code.

Finally, measure before and after: the time until the first words appear, and the time until the answer is complete. Report both numbers to me.

Conclusion

Why do AI answers need to be streamed? Because people want answers in under a second, and AI cannot write a full answer that fast today. On a website, the answer travels from the browser to your server to the AI and back, and if every step waits for the whole answer, the page looks broken. Tune the model so it answers sooner, use AI only for questions that need it, and stream the rest so the first words appear right away. Your visitors will not notice that the answer took ten seconds. They will notice that it started in one.

For more on putting AI to work on your website, read Why Websites Need Three Interfaces and How to Build an AI Agent.

Want to see a streamed answer? Ask the chat on this page a question and watch it appear. Then run MachineReady on your website to see how AI reads your site, or send our A2A agent to your site. Need help making your own AI feature fast? We build AI agents for clients, or get in touch.

← Back to all articles