Streaming & Progress

Long calls needn’t be opaque. When the client attaches a progressToken and the server advertised streaming, the server MAY emit notifications/progress frames before the final response — reporting pipeline stages and even partial hits.

Opting in

The client sets _meta.progressToken on the request. The token is a client-chosen string or integer, unique among the client’s in-flight requests.

{
  "jsonrpc": "2.0", "id": 2, "method": "retrieve",
  "params": { "query": "…", "k": 5, "_meta": { "progressToken": "t2" } }
}

Progress frames

Each frame is a notification (no id, never answered):

{ "jsonrpc": "2.0", "method": "notifications/progress",
  "params": { "progressToken": "t2", "progress": 0.6, "stage": "rerank",
              "message": "reranked 30/50", "partial": { "hits": [] } } }

string | integer

Echoes the request’s token verbatim. A server MUST NOT emit progress for a request that carried none.

float

Completion in 0.0–1.0.

string

The current pipeline stage (transform, recall, rerank, mmr, …).

object

Optional incremental payload — e.g. partial.hits with first-stage candidates before rerank.

The final response is authoritative

Frames are advisory

A client MUST function correctly even if it never receives a single progress frame. Never build correctness on them.

Ignore unknown tokens

A client MUST ignore a frame bearing a token it doesn’t recognise.

The result still arrives

The normal JSON-RPC response bearing the request id still follows every stream and is the single source of truth.

Over HTTP: SSE

Over stdio, progress frames simply interleave on stdout. Over HTTP, a client that sends Accept: text/event-stream receives them as Server-Sent Events — zero or more notifications/progress frames, then one final frame carrying the JSON-RPC response. See Transports. Each SSE frame is a single data: line whose payload is a compact JSON object, terminated by a blank line; the stream ends after the final response frame. A client that did not ask for SSE gets one buffered JSON response instead — so the same server handler serves both callers.

Writing a streaming handler

The reference Python SDK turns this into one idiom: a generator handler that yields rcp.Progress events and returns the final result. Register it with Server.stream(...) and advertise streaming. serve_http flushes each Progress as a notifications/progress frame the instant it is produced, then sends the final response frame; a plain unary POST drains the same generator and returns only the result.

import rcp

def retrieve(params):
    query, k = params["query"], params.get("k", 5)
    dense  = recall_dense(query)
    yield rcp.Progress(0.33, "recall:dense", partial=[h["id"] for h in dense[:k]])

    fused  = rcp.rrf_fuse({"dense": dense, "sparse": recall_sparse(query)}, k=k * 2)
    yield rcp.Progress(0.66, "fuse:rrf", partial=[h["id"] for h in fused[:k]])

    hits   = rerank(query, fused)[:k]
    yield rcp.Progress(1.0, "rerank", partial=[h["id"] for h in hits])
    return {"hits": hits, "mode": "hybrid"}

s = rcp.Server()
s.set_info("streaming-demo", "1.0")
s.advertise(rcp.Capability.Retrieve, {"maxK": 100, "modes": ["hybrid"]})
s.advertise(rcp.Capability.Streaming)   # required for the SSE path
s.stream("retrieve", retrieve)
s.serve_http(8000)

frame

A progress event. progress is 0.0–1.0; stage names the pipeline phase; partial is an optional early-result payload. Serialised into a notifications/progress frame carrying the request’s progressToken — emitted only when the client supplied one.

The handler is written once and works both ways. Over SSE the frames stream live; over a plain request the generator is drained and only its return value is sent — satisfying §13’s rule that a non-SSE client MUST get a single buffered JSON response.

A complete, self-contained program — a real HTTP+SSE server plus a raw-socket client that prints each frame as the retrieve funnel fills — lives at examples/example_streaming.py:

python3 examples/example_streaming.py
#   progress  33%  recall:dense   partial=['d2', 'd5', 'd1']
#   progress  66%  fuse:rrf       partial=['d2', 'd5', 'd1']
#   progress 100%  rerank         partial=['d2', 'd5', 'd1']
#   final response frame: hits = ['d2', 'd5', 'd1'] …

Progress pairs naturally with cancellation: stream partial hits, let the user act on them, and send notifications/cancel if they navigate away.

Get Started

Core Concepts

Methods

Features

Operating a Server

Opting in

Progress frames

The final response is authoritative

Over HTTP: SSE

Writing a streaming handler

​Opting in

​Progress frames

​The final response is authoritative

​Over HTTP: SSE

​Writing a streaming handler

Opting in

Progress frames

The final response is authoritative

Over HTTP: SSE

Writing a streaming handler