fidius-test::stream
Rust
Composition harness for streaming plugins — for tests, not production.
stream_of / collect / pump make it trivial to test a streaming plugin
in isolation or to wire a producer to a consumer. They are the reference |
for “pipes of plugins.”
Stability
This module is deliberately NOT part of fidius’s semver-stable surface.
It exists so tests can compose streams without re-deriving the same loop, and
so the canonical composition pattern is readable in one place. Its behaviour
is correct (pull-paced backpressure, stop-and-surface on the first error)
but its API may change at any time. In production, write your own pump
loop — fidius ships the typed pipe (the ChunkStream primitive), not an
orchestrator; orchestration (scheduling, retries, observability) is yours.
See ADR FIDIUS-A-0004 (“Streaming as Mechanism, Not Protocol”).
Structs
Section titled “Structs”struct CollectSink
Section titled “struct CollectSink”pub
Derives: Default
A StreamSink that records everything it accepts — for asserting on the far end of a pump.
Methods
Section titled “Methods”pub
fn new() -> SelfA fresh, empty sink.
Source
pub fn new() -> Self { Self::default() }pub
fn take(&self) -> Vec<Value>Snapshot of everything accepted so far.
Source
pub fn take(&self) -> Vec<Value> { self.items.lock().unwrap().clone() }Traits
Section titled “Traits”trait StreamSink
Section titled “trait StreamSink”pub
The destination side of a pipe: a consumer pump hands each item to, in pull order. Deliberately tiny — a real destination plugin presents whatever richer surface it likes; this is only what composition tests need.
Methods
Section titled “Methods”accept
Section titled “accept”async
async fn accept(&self, item: Value) -> Result<(), CallError>Accept one streamed item. Returning Err stops the pump and surfaces.
Source
async fn accept(&self, item: Value) -> Result<(), CallError>;Functions
Section titled “Functions”fn stream_of
Section titled “fn stream_of”pub
fn stream_of(items: Vec<Value>) -> ChunkStreamAn in-memory source over a fixed item sequence. Test-only by nature — you would never construct a stream from a Vec in production; that it is useful only in tests is exactly why the whole module is test-tier.
Yields each value as Ok, then a clean end of stream.
Source
pub fn stream_of(items: Vec<Value>) -> ChunkStream { ChunkStream::new(futures::stream::iter( items.into_iter().map(Ok::<Value, CallError>), ))}fn collect
Section titled “fn collect”pub
async fn collect(mut s: ChunkStream) -> Result<Vec<Value>, CallError>Drain a stream to a Vec, stopping at — and returning — the first error. The single-plugin unit-test idiom: collect(plugin.process(stream_of(rows))).
Source
pub async fn collect(mut s: ChunkStream) -> Result<Vec<Value>, CallError> { let mut out = Vec::new(); while let Some(item) = s.next().await { out.push(item?); } Ok(out)}fn pump
Section titled “fn pump”pub
async fn pump<S>(mut out: ChunkStream, into: &S) -> Result<(), CallError>where S: StreamSink + ?Sized,The reference pull-loop wiring a producer stream to a StreamSink.
Pull-paced: exactly one item is awaited at a time, so a slow sink naturally
backpressures the producer (the next item is not pulled until the sink has
accepted the current one). Stops at the first error from either side —
producer error or sink rejection — and returns it; on a clean end of stream
returns Ok(()). This is the ~10 lines you would copy into production and
then grow your own retries/observability around.
Source
pub async fn pump<S>(mut out: ChunkStream, into: &S) -> Result<(), CallError>where S: StreamSink + ?Sized,{ while let Some(item) = out.next().await { into.accept(item?).await?; } Ok(())}