Mira quality system

Evaluation console

Use the smaller release check for routine validation, resume an interrupted run with Run remaining, or run the complete Golden Dataset for a full regression audit. Results persist in this browser. Compare models remains an evaluation-only experiment.

Loading the assistant engine...

What the live bot records

The portfolio sends one anonymous portfolio_assistant_response event per completed request. Email addresses and phone numbers are redacted from questions before capture. Use the request ID to join helpfulness and downstream conversion events.

total_latency_msUse for P50 and P95 latency.

generation_latency_msTime spent in model generation. Deterministic and guardrail answers show zero.

input_tokens_estimateApproximate input tokens based on characters.

output_tokens_estimateApproximate generated answer tokens.

estimated_cost_usdHosted-model reference estimate; browser-hosted Qwen comparisons are zero.

retrieval_cache_hitWhether an approved evidence-bound cache answered before Gemini was called.

generation_cache_hitWhether a validated near-identical question reused an answer whose approved sources are still present.

retrieved_sourcesPortfolio sources supplied to the answer path.

fallbackWhether a safe fallback was returned.

errorGeneration or request failure, when present.

portfolio_assistant_helpfulnessYes or no rating joined by request ID.

portfolio_conversionResume, case study, and contact clicks.

Telemetry guidePostHog dashboard recipe and event dictionary.