Durable generation

The assistant from Streaming AI responses stops when the shopper leaves, which is right for a quick question. For a long summary that should finish even if the tab closes, and still be there on reload, the generation has to outlive the request.

In the streaming guide, the request owns the generation, and closing the page cancels it. On this page, a job on the server owns it. A server function starts the job and returns. The job writes the reply into storage as the model produces it. Every page that shows the reply reads it from storage through a live source.

Use the streaming guide's approach when only one reader needs the answer and an abandoned answer is worthless. Use this approach when the reply must finish without its reader, be there on reload, or be read from more than one tab.

This page shows two ways to deliver the reply:

  • In the browser: the reply is sent as data through live() and rendered on the client.
  • On the server: the whole thread renders on the server as a live server component, as in the chat example in the Solid repository. Server components are an experimental preview.

The job, the storage, and the server functions are the same in both.


The job

The job has three parts: a store the job writes into, a registry of running jobs that no request owns, and the server functions that start and stop a job.

The store keeps each reply and tells watchers when it changes:

src/server/replies.ts
export type Reply = {
id: string;
text: string;
status: "generating" | "complete" | "stopped" | "failed";
error?: string;
};
const replies = new Map<string, Reply & { version: number }>();
const waiters = new Map<string, Set<() => void>>();
export function write(id: string, patch: Partial<Reply>) {
const current = replies.get(id)!;
replies.set(id, { ...current, ...patch, version: current.version + 1 });
const waiting = waiters.get(id);
waiters.delete(id);
waiting?.forEach((wake) => wake());
}
function changed(id: string, signal?: AbortSignal) {
return new Promise<void>((wake) => {
(waiters.get(id) ?? waiters.set(id, new Set()).get(id)!).add(wake);
signal?.addEventListener("abort", () => wake(), { once: true });
});
}
// The reply as it stands now, then again after every write.
export async function* watch(id: string, signal?: AbortSignal) {
let seen = -1;
while (!signal?.aborted) {
const current = replies.get(id);
if (current && current.version !== seen) {
seen = current.version;
yield current as Reply;
continue;
}
await changed(id, signal);
}
}

watch checks the version after every yield before it waits, so a write made while the consumer is still handling the previous value is not missed. The chat example keeps the same shape in src/server/db.ts, written through to disk so a reply survives a restart.

The job reads from the model and writes into the store. It belongs to the module, not to a request:

src/server/generations.ts
import * as replies from "./replies";
const running = new Map<string, AbortController>();
export function start(replyId: string, history: Message[]) {
const abort = new AbortController();
running.set(replyId, abort);
void (async () => {
let text = "";
try {
for await (const chunk of model.stream({
history,
signal: abort.signal,
})) {
text += chunk.text;
replies.write(replyId, { text });
}
replies.write(replyId, { status: "complete" });
} catch (error) {
if (abort.signal.aborted) replies.write(replyId, { status: "stopped" });
else
replies.write(replyId, {
status: "failed",
error: "The model stopped answering",
});
} finally {
running.delete(replyId);
}
})();
}
export function stop(replyId: string) {
running.get(replyId)?.abort();
}

model.stream is a stand-in for the provider SDK, as in the streaming guide. The signal it receives is the job's own, not a request's: Stop aborts it, and nothing else does. A reply that fails keeps the text that had arrived, with the reason beside it; a reply that was stopped keeps its text and says so. Write at most every few tens of milliseconds when the model is fast, as the example does, since every write wakes every watcher.

The server functions start and stop a job and return at once:

src/data/replies.ts
import { formAction } from "@solidjs/router";
import * as generations from "../server/generations";
import * as replies from "../server/replies";
import * as threads from "../server/threads";
export async function sendMessage(
threadId: string,
messageId: string,
text: string
) {
"use server";
const { replyId, history } = await threads.append(threadId, messageId, text);
generations.start(replyId, history);
}
export const stopReply = formAction(async (replyId: string) => {
"use server";
generations.stop(replyId);
}, "stopReply");

sendMessage stores the shopper's message and an empty reply with status generating, starts the job, and returns. The response closes while the job is still writing. The server-function runtime does not stop work that a function started, so the job runs to the end whether or not anyone is watching. threads.append is the thread half of the store, which keeps the message list and the history the model reads; the chat example's db.send is one. stopReply is a Solid Router form action, so the Stop button works as a form before JavaScript has loaded. sendMessage is a plain server function that the composer calls from code; the router handles its response, including revalidation, the same way it does for a form.

Mark replies that a restart interrupted

A server restart ends every running job and leaves the replies they were writing at generating. Mark them failed with a reason when the store loads, as the chat example does, so a reader sees a message with a Retry rather than a reply that never finishes.


Render the reply in the browser

The live source yields the reply so far, whole, on every write:

src/data/replies.ts
import { getRequestEvent } from "@solidjs/web";
import { live } from "@solidjs/web/server-functions";
import * as replies from "../server/replies";
export const watchReply = live(async function* (replyId: string) {
"use server";
const signal = getRequestEvent()?.request.signal;
for await (const reply of replies.watch(replyId, signal)) yield reply;
});

The request's signal ends the watcher when the reader leaves; the job does not see it. The first yield on every connection is the current reply, which live() relies on when it reconnects, as Declare a live source describes.

In the browser, createProjection with an async generator turns the stream into a store:

src/pages/Reply.tsx
import { Show, createProjection } from "solid-js";
import { stopReply, watchReply } from "../data/replies";
type ReplyView = { text: string; status: string; error?: string };
function Reply(props: { replyId: string }) {
const reply = createProjection(
async function* (draft: ReplyView) {
for await (const next of watchReply(props.replyId)) {
draft.text = next.text;
draft.status = next.status;
draft.error = next.error;
yield;
}
},
{ text: "", status: "generating" },
{ seedLoadingValue: true }
);
return (
<article>
<p>{reply.text}</p>
<Show when={reply.status === "generating"}>
<form action={stopReply.with(props.replyId)} method="post">
<button type="submit">Stop</button>
</form>
</Show>
<Show when={reply.status === "failed"}>
<p role="alert">{reply.error}</p>
</Show>
</article>
);
}

Send a message, close the tab while the reply is being written, and open the page again. The component calls watchReply again, the first yield is the reply so far, and the paragraph keeps growing from there. Open the page in two tabs and both show the same reply; press Stop in one and the status changes in both, because Stop aborted the job and the job wrote stopped into the store every watcher reads.

The projection reads props.replyId before its first await, so a different reply re-runs it, which ends the previous iteration and its connection. A projection is used instead of a memo so that readers of reply.status do not update on every text change, and so the Markdown block split from Render Markdown without re-rendering the answer has a draft to write into. refresh(reply) re-runs the projection, which opens a new connection; it does not touch the job.

Each place that reads a live source opens its own connection. When the thread list and the thread both need the same reply, move the read up to a shared parent, or wrap it in Solid Router's liveQuery, which keeps one connection per key for every reader: liveQuery(watchReply, "reply"). liveQuery marks the server function live() itself; a function already marked live() is unchanged.


Show the message until the stream confirms it

The shopper's own message should be on screen the moment they press Enter, and should give way to the server's row when the thread shows it. The thread is read through a live source, so the confirmation arrives on the stream rather than in the action's response. That is what until waits for:

src/components/Composer.tsx
import { For, action, createOptimisticStore, until } from "solid-js";
import { sendMessage } from "../data/replies";
type Pending = { id: string; text: string };
function Composer(props: { threadId: string; seen: string[] }) {
const [outbox, setOutbox] = createOptimisticStore<Pending[]>([]);
const submit = action(function* (text: string) {
const id = crypto.randomUUID();
setOutbox((list) => {
list.push({ id, text });
});
yield sendMessage(props.threadId, id, text);
yield until(() => props.seen.includes(id), { timeout: 15_000 });
});
return (
<>
<For each={outbox.filter((m) => !props.seen.includes(m.id))}>
{(message) => <p class="pending">{message.text}</p>}
</For>
<form
onSubmit={(event) => {
event.preventDefault();
const data = new FormData(event.currentTarget);
void submit(String(data.get("text") ?? ""));
}}
>
<textarea name="text" required />
<button type="submit">Send</button>
</form>
</>
);
}

seen is the list of message ids the live thread shows; the parent reads it from the thread's live source and passes it down. The push onto outbox is an optimistic write: it shows at once and lasts until the action finishes. until reads the real seen list, not the optimistic one, so only the stream can satisfy it. When the server's row arrives, until resolves, the action finishes, and the optimistic message is removed. A timeout or a server error rejects the action, which also removes the optimistic message and leaves the shopper's text to send again.

The For also filters the outbox by seen. The optimistic message is removed one task after the server's row arrives, so without the filter one frame can show both. Filtering by seen closes that gap, and until still ends the action on failure or timeout.

action here is the one from solid-js, which runs a generator as one transaction. sendMessage is the server function itself, called directly, and the router still applies its revalidation and any redirect.


Render the thread on the server

Server components are an experimental preview

A function returned from a "use server" function is a server component, mounted in the browser with dynamicComponent. The runtime ships in @solidjs/web and is excluded from Solid 2.0's stability guarantees: the API and the wire format may change in any release. The chat example in the Solid repository is this section's reference implementation.

To keep the Markdown renderer, the highlighter, and the model client on the server, render the thread on the server and keep updating it. The server function returns a component instead of data, and liveQuery, which marks it live(), keeps the response open while it re-renders:

src/routes/thread.tsx
import { createMemo, onCleanup } from "solid-js";
import { getRequestEvent } from "@solidjs/web";
import type { Slot } from "@solidjs/web/frames";
import { liveQuery } from "@solidjs/router";
import { stopReply } from "../data/replies";
import * as threads from "../server/threads";
type ComposerSlot = Slot<{ threadId: string; seen: string[] }>;
export const getThread = liveQuery(async (id: string) => {
"use server";
return (props: { composer: ComposerSlot }) => {
const gone = new AbortController();
onCleanup(() => gone.abort());
getRequestEvent()?.request.signal.addEventListener("abort", () =>
gone.abort()
);
const view = createMemo(() => threads.watch(id, gone.signal));
return (
<article>
{view().messages.map((message) => (
<section data-id={message.id}>
<div innerHTML={renderMarkdown(message.text)} />
{message.status === "generating" && (
<form action={stopReply.with(message.id)} method="post">
<button type="submit">Stop</button>
</form>
)}
</section>
))}
<props.composer
$key={id}
threadId={id}
seen={view().messages.map((m) => m.id)}
/>
</article>
);
};
}, "thread");

threads.watch is the thread-level watcher: it yields the thread and its messages after every write to any of them. The route component mounts the result and passes in the composer:

src/routes/thread.tsx
import { Loading } from "solid-js";
import { dynamicComponent } from "@solidjs/web";
import type { RouteSectionProps } from "@solidjs/router";
import Composer from "../components/Composer";
export default function ThreadRoute(props: RouteSectionProps) {
const Thread = dynamicComponent(() => getThread(String(props.params.id)));
return (
<Loading fallback={<p>Loading the conversation…</p>}>
<Thread
composer={(p) => <Composer threadId={p.threadId} seen={p.seen} />}
/>
</Loading>
);
}

The server places the composer from the previous section at the bottom of the thread and passes it seen, so until reads the same ids the server rendered. $key={id} makes a new composer per thread, so a half-typed message stays with its conversation.

What this shape adds over the client-owned one:

  • The initial HTML already contains the reply so far, because the server renders the live source's first value; hydration reuses it and then connects.
  • The browser bundle has no Markdown parser, no highlighter, and no model client.
  • Every mutation is a server-rendered form, which works before JavaScript loads; a submitted form stays marked busy while the markup around it changes.

What it costs: the server holds one render per open thread per tab, and the thread and the sidebar are two connections per tab, which is why the example serves HTTP/2 in development.

The example reads the thread through one watcher at the top of the render and splits the markup into finished messages and the reply being written, so each new token re-sends only the reply. Do not create a watcher per message inside the list: each re-render makes new ones, so the render never finishes.


Reconnect, and where a reply resumes from

If a stream drops after it connected, live() reconnects with exponential backoff, and the first yield of the new connection replaces what the page shows. Each event on the wire carries an id, and every reconnect sends the last id it received back as Last-Event-ID. When that id matches the hash of the source's first yield, the server skips that yield because the client already has it, so an unchanged reply costs nothing on reconnect. For a live server component, the reconnect also lists the pieces of markup the page shows, and the server sends only the pieces that changed. After hydration, the first connection names the value the page was served with, so the server sends nothing if the value is unchanged.

A reconnect does not send only the part of the reply written since the last event. The source yields the whole reply, so each reconnect sends the full reply so far. To send only the new part, keep an append-only log of chunks per reply, read the header in the source with getRequestEvent()?.request.headers.get("last-event-id"), and yield the chunks after that position. The hash skip does not apply to a source that reads the header itself. @tanstack/ai-durable-stream and Electric's Durable Streams are off-the-shelf logs of that kind, for a team that wants one rather than a table.


Common problems

The reply stops when the shopper leaves

The job received the request's signal instead of its own. Give the job an AbortController the registry owns, and pass request.signal only to the watcher.

Stop does nothing

The Stop button aborted a controller the job was not reading, or the job was started by a server that has since restarted. Check that generations.start passes abort.signal to the model call, and that the store marks orphaned replies failed on load.

Two tabs show different replies

Each tab is reading a copy the job made rather than the store. The job writes once, into the store; every page reads the store through its watcher.

The bubble and the server's row show together for a frame

The optimistic message is removed when the action finishes, one task after the server's row arrives. Filter the outbox by seen as well as waiting on it with until.

Cannot call until inside a reactive scope

until was called inside a memo or an effect. Await it from an action body or another imperative scope.


Recap

  • Start the generation from a server function that returns at once; the job belongs to a module-level registry and writes the reply into a store as it arrives.
  • Pass the job its own abort signal; the request's signal ends only the watcher.
  • Deliver the reply through live(async function*), yielding the whole reply so far and the current reply first on every connection.
  • Render it in the browser with createProjection, or on the server as a live server component mounted with dynamicComponent through liveQuery(fn, key).
  • Show an optimistic message with createOptimisticStore inside an action, wait for it with until on the live thread's ids, and filter it by the same ids.
  • A reconnect sends Last-Event-ID; an unchanged first value is skipped, and sending only new text needs a log the source reads the header against.

Next steps

Last updated: 11/17/26, 5:22 PMEdit this pageReport an issue with this page