Under the hood · the private pipeline

Watch your message flow through 12 stages

Every Pro request runs the same guarded path — redacted, routed, answered, scored. Follow the packet as it moves, see what runs in parallel, and how each answer quietly makes the next one better.

Packet:📥 raw input
You paste into Lyra chat →
“My name is John Smith, email john@acme.com — summarise this quarter's sales.”
📎 receipt.jpg · image inputmultimodal — coming to chat
① Ingress — always first
🛡️
01PII Guard
Presidio

Names, emails, cards → redacted before anything else touches the text

② Safety gate
🚦
02Safety Check
LLM-Guard

Prompt-injection & unsafe content gate — blocks here or proceeds

③ Pre-flight — runs in parallel
These four kick off together on the cleaned text
⇄run concurrently
⚡
03Semantic Cache
Redis

Seen a near-identical ask? Return instantly, skip the model entirely

🗜️
04Token Compress
LLMLingua

Prunes the prompt ~2–3× — fewer tokens, lower cost, faster

🧠
05Memory Inject
Mem0 + Chroma

Pulls your relevant past context so the answer knows you

🧭
06Model Router
LiteLLM

Classifies intent → picks the cheapest capable model (local vs cloud)

④ Generate
🤖
07LLM Inference
Ollama / Cloud

The actual generation — the one expensive step on the hot path

✓ Your answer is delivered here
Everything below runs in the background — you never wait for it
⑤ Deliver — your answer arrives here
🧹
08PII Scrub Output
Presidio

Scrubs any personal data out of the reply — then it's yours

⑥ Learn — async, after you're answered
Off the hot path — you never wait for these
⇄deferred · async write-back
✅
09Response Validation
Ragas → QV-RMasync

Scores quality → the reward signal that trains the whole pipeline

💾
10Memory Store
Chromaasync

Saves this turn so future answers keep your context

📥
11Cache Store
Redisasync

Caches the answer so the next similar ask is instant

📒
12Audit Log
Postgresasync

Immutable, PII-free record — model, cost, quality, versions

Speed by design

Parallel where it can be, deferred where it should be

Not every stage waits its turn. The hot path is kept short — the write-back tail is pushed off it entirely.

🔒
1 step is truly blocking-first
PII redaction (Stage 1) must run before anything else sees the text. Non-negotiable — it's the privacy guarantee.
⇄
4 pre-flight stages run concurrently
Cache lookup, Compress, Memory inject and Route all read the same cleaned text — they fan out in parallel instead of queuing.
🤖
1 expensive step: inference
The model call is the real cost on the hot path. A cache hit or a local route can skip or shrink it.
📤
You get your answer at Stage 8
After the output scrub, the reply is yours. Perceived latency ends here.
♻️
4 write-tail stages go async
Validate, Memory-store, Cache-store and Audit run after delivery — fire-and-forget, off your critical path.
Net effect: you wait for redaction → parallel pre-flight → one model call → output scrub. The four scoring/storage stages happen after you already have your answer.
It learns from itself

A reward model at Stage 9

Response validation isn't just a pass/fail. It emits a quality score — the reward signal that tunes routing, caching and compression, blending reinforcement learning with supervised human calibration.

♻️
Reinforcement
Every answer becomes a reward

Stage 9 scores each finished run (task-completion, grounding, safety, persona-fit, conciseness). That composite reward is fed back as a bandit signal to the Router (which model to pick), the Cache (spot quality drops on a hit) and Compress (how hard to prune).

🎓
Supervised
Humans set the gold standard

A calibrated set of 200 human-rated transcripts anchors the scorer. Lowest-confidence runs + every hard-fail are routed to a review UI; corrections flow back into the gold set and a weekly LoRA refresh distils the judge into a fast on-box scorer.

🛡️
Hard-fail override
Safety can't be averaged away

Any tier violation, PII leak past scrub, or hallucinated action zeroes the reward and pages a human — it never quietly averages into a 'good enough' score.

✅ Stage 9 · quality.scored──▶🧭 Router reward⚡ Cache quality-gate🗜️ Compress arm
👤 Human review · 200 gold set──▶🔁 Weekly LoRA refresh──▶⚡ Fast on-box scorer

Runs async on PII-scrubbed transcripts only · every score is auditable (model + rubric version + input hash)

Private by construction, smarter every day

Local is free forever. Pro adds the full guarded pipeline.

⬇ Download Lyra — free← Back to Lyra