Skip to content

MVP Limitations

ReplayLab's current MVP is local-first and intentionally narrow. These boundaries keep capture, replay, comparison, and generated regressions deterministic while the core loop stabilizes.

Supported Provider Paths

  • OpenAI Responses: non-streaming OpenAI().responses.create(...).
  • HTTP clients: sync requests and sync httpx.
  • Capture modes: explicit same-process instrumentation and Python child-process auto-patching through replaylab run.
  • Replay modes: same-process adapters and Python child-process auto-patching through replaylab replay.

Full-Payload Requirement

Replay can inspect metadata-only capsules, but it cannot serve provider responses from them. Use full payload capture when a capsule must be replayed or converted into a pytest regression:

uv run replaylab run \
  --capture-payload-policy full \
  --auto-patch-integrations openai,requests,httpx \
  -- python app.py

Generated pytest regressions also require succeeded provider boundaries with response payload refs.

Wrapper And Child Capsules

replaylab run currently writes separate artifacts:

  • a wrapper capsule for command metadata and exit status
  • a child provider capsule for captured OpenAI, requests, or httpx boundaries

Use replaylab capsule list to find the child provider capsule before replaying or generating tests. Parent/child capsule merging is not implemented yet.

Not Implemented Yet

  • OpenAI streaming responses.
  • OpenAI Chat Completions.
  • async httpx.
  • file uploads, multipart requests, streaming HTTP bodies, and streaming downloads.
  • header capture by default.
  • cloud upload, hosted issue grouping, and web UI.
  • parent/child record streaming or merged wrapper capsules.
  • perturbation mode.
  • failed-boundary regression generation.
  • report-to-report comparison.

Secret Safety

CLI output and inspection output should not print payload bodies, API keys, or raw secret values. Payload redaction happens before payload bytes are written when a redaction policy is configured.