# Batching

> Every question created in the same synchronous turn travels in one request. What that means, how to keep it, and what it saves.

Section: Concepts · HTML: https://jevascript.org/docs/batching · Markdown: https://jevascript.org/docs/batching.md

## One turn, one request

A context queues each question and flushes the queue on the next microtask. Everything created before the current synchronous turn ends goes out together:

```ts
const t = await semantic(ticket).batch({
  urgency: score("A human needs to act on this urgently."),
  frustration: score("The person writing this sounds frustrated or angry."),
  security: is("This describes a security or access-control problem."),
  team: choose(["billing", "technical", "security"]),
})
// 1 request, 4 answers
```

`batch()` is the explicit form: it names the results and gives you a typed object. `Promise.all` on the same context batches identically:

```ts
const s = semantic(ticket)
const [urgency, frustration] = await Promise.all([
  s.score("A human needs to act on this urgently."),
  s.score("The person writing this sounds frustrated or angry."),
])
// also 1 request
```

### What cannot batch

Sequential `await`s cannot share a request. The second question does not exist until the first has resolved:

```ts
const s = semantic(ticket)
const urgency = await s.score("...")       // request 1
const frustration = await s.score("...")   // request 2
```

When this happens outside production, the runtime warns once per context:

```text
[semantic] This context issued a second provider request. Questions awaited one at a time cannot share a request. Use `batch({...})` or `Promise.all([...])` to send them together — batching is dramatically cheaper and no slower.
```

Turn it off with `warnUnbatched: false`, or route it to a hook with `observability.onUnbatched`. See [Configuration](https://jevascript.org/docs/configuration).

## What it saves

Measured against the live API on a short support ticket with seven questions, median of three runs (2026-09):

|  | Batched | Sequential |
| --- | --- | --- |
| Requests | 1 | 7 |
| Tokens | 490 | 2440 |
| Latency | 801 ms | 2572 ms |

**5.0× cheaper and 3.2× faster** for the same seven answers. The saving grows with the size of the state, because a sequential call re-sends the whole state every time. On a long document it approaches a full N×.

## How it works

1. `is()`, `score()` and `choose()` each push a planned question onto the context and return a promise.
2. The first push schedules a flush with `queueMicrotask`.
3. When the flush runs, pending questions are grouped by provider and sent as one `SemanticRequest` per provider.
4. Every promise settles from the same response.

Two details follow from this:

- **Timeouts.** When questions with different `timeoutMs` share a request, the largest applies to the request.
- **Providers.** A question with its own `provider` option joins a separate request to that provider, in the same turn.

## Batching inside definitions

A [schema](https://jevascript.org/docs/definitions) resolves every field in one request, however many there are. A defined metric or rule exposes `.question()` so it can join a batch with ad-hoc questions:

```ts
const t = await semantic(ticket).batch({
  churn: churnRisk.question(),          // a defineMetric
  safe: safeToAutoReply.question(),     // a defineRule
  team: choose(["billing", "technical"]),
})
```

## Batching across items

Batching is about one state and many questions. Many states are a different problem: each item is its own request, because a request carries one state. [Collections](https://jevascript.org/docs/collections) explains which operations cost one request and which cost N, and the [RAG gate](https://jevascript.org/docs/scenarios/rag-gate) scenario shows how to fold many items into one state when they are small enough.

## Proving it in tests

The mock provider counts requests. This assertion is the one that catches a refactor that broke batching:

```ts
const provider = createMockSemanticProvider({ urgently: 0.9, frustrated: 0.7 })
const semantic = createSemantic({ provider })

await semantic(ticket).batch({ urgency: score("..."), frustration: score("...") })

assert.equal(provider.requestCount, 1)
```