Building Generative AI Apps with React

Text generation apps look simple in a demo and get annoying fast in production. The model is slow, output arrives in fragments, and users will mash the button. Here is the setup we use for a basic completion UI, and where it bites.

What you need

• React, with hooks. Nothing exotic.
• An OpenAI API key, kept on the server. Never in the client bundle.
• Node 18 or later, for native fetch and streams.

The client

You can get surprisingly far with plain fetch and a reader loop. Vercel's AI SDK wraps this up in useCompletion, and we use it on client work, but you should know what it does underneath.

import { useState } from 'react';

function AIWriter() {
  const [prompt, setPrompt] = useState('');
  const [completion, setCompletion] = useState('');

  const generateText = async () => {
    setCompletion('');
    const res = await fetch('/api/generate', {
      method: 'POST',
      headers: { 'Content-Type': 'application/json' },
      body: JSON.stringify({ prompt }),
    });

    const reader = res.body.getReader();
    const decoder = new TextDecoder();
    while (true) {
      const { done, value } = await reader.read();
      if (done) break;
      setCompletion(prev => prev + decoder.decode(value, { stream: true }));
    }
  };

  return (
    <div>
      <textarea value={prompt} onChange={e => setPrompt(e.target.value)} />
      <button onClick={generateText}>Generate</button>
      <div className="output">{completion}</div>
    </div>
  );
}

The server

The route handler forwards the prompt to OpenAI with stream: true and pipes the response body straight back to the client. In Next.js that is a route handler returning a ReadableStream. The call lives here so the key stays on the server, and set a timeout, because the default is to hang forever.

Where it goes wrong

1. Latency. First token in under a second feels fine. Eight seconds of nothing followed by a complete answer does not. Stream, always.
2. Chat history. Keeping every turn in React state is easy. Sending every turn back to the model is how you blow the context window and the invoice. Truncate or summarize old turns server side.
3. Markdown. Models emit it whether you asked or not. react-markdown handles it, but it re-parses on every token, so throttle updates on long outputs.

Where this leaves you

That is the whole core of a generation UI: a POST, a reader loop, some state. The libraries add aborts, retries, and chat plumbing, and they are worth it once you move past a demo. Learn the loop first so you can debug them when they misbehave.