Runtime Intelligence
for AI Agents
Critiqor evaluates observable runtime behaviour rather than relying on agent self-reporting. Capture runtime evidence. Generate explainable diagnoses. Improve agent reliability.
$ pip install critiqor- step 01DeveloperCLI
- step 02AI AgentOpenClaw
- step 03Runtime Eventsobserved
- step 04Evidence Collectiontool calls · outputs
- step 05Diagnosis Engineexplainable
- step 06Interactive Dashboardverdict · timeline
Traditional evals judge the answer.
Critiqor watches the work.
Most evaluation frameworks score the final response. Critiqor instead observes the agent during execution — recording tool calls, tool outputs, runtime events, reasoning flow, execution efficiency and evidence utilisation.
Every diagnosis is backed by observable execution evidence — not a model's opinion about itself.
Answer-only scoring
- Final response only
- Self-reported reasoning
- Limited explainability
- No runtime visibility
Evidence-backed diagnosis
- Runtime evidence
- Observable execution
- Explainable diagnosis
- Root cause analysis
- Historical intelligence
Everything you need to trust your agents.
Six capabilities working together — from the moment your agent boots to the final diagnosis.
Runtime Observation
Observe agents during execution — not after. Critiqor attaches before launch and follows every event end-to-end.
Evidence Collection
Capture runtime events, tool calls, tool outputs, provider requests and rich execution metadata into structured artifacts.
Diagnosis Engine
Convert raw runtime evidence into explainable reports — verdicts, confidence, trust score and reasoning summaries.
Root Cause Analysis
Identify failures, ignored outputs, loops and retrieval gaps. Each issue links back to the underlying evidence.
Interactive Dashboard
Executive summaries, runtime timelines, causal graphs and historical evaluations — local-first, no data leaves your machine.
Benchmarking & Leaderboards
Compare agent reliability with private, anonymous and public visibility. Opt-in benchmarks for teams and communities.
How Critiqor works.
A nine-stage pipeline turning raw agent execution into evidence, diagnosis and recommendations — fully local by default.
Developer Workflow.
Four commands from install to insight. Local. Reproducible. Friction-free.
Install Critiqor
One pip install. Zero infrastructure. No accounts, keys or cloud services required.
$ pip install critiqorLaunch under observation
Critiqor launches OpenClaw and immediately begins observing runtime activity.
$ critiqor monitor openclawUse OpenClaw normally
Work as you always do. Critiqor observes silently in the background — zero changes to your agent code.
› openclaw run … # business as usualFinalize the session
Critiqor finalizes the observation session, generates the diagnosis and automatically opens the local dashboard.
$ critiqor finalizeRead the evidence. Trust the verdict.
A local-first interface designed for engineers — fast, dense, explainable.
Dashboard
local diagnosisLive reliability intelligence for OpenClaw agents.
Runtime evidence captured
No OpenClaw failure mode was detected from runtime evidence.
- passedopenclaw_agent runtime runrun_004 · openclaw
- passedopenclaw_agent runtime runrun_003 · openclaw
- passedopenclaw_agent runtime runrun_002 · openclaw
- passedopenclaw_agent runtime runrun_001 · openclaw
Install Critiqor. Observe everything.
One pip install, three commands. No accounts. No cloud. Just runtime evidence.
$ pip install critiqor$ critiqor monitor openclaw$ critiqor finalizeBuilt in the open, with you in the loop.
Transparent roadmap, public issues, and fast iteration on real agent failures.
- Runtime Observation
- Interactive Dashboard
- OpenClaw Integration
- Diagnosis Engine
- Root Cause Analysis
- Improved benchmarking — compare agents across runs and environments.
- Dashboard enhancements — denser evidence views for faster debugging.
- Additional agent frameworks — extend beyond OpenClaw while keeping local-first.
- Community leaderboards — opt-in reliability benchmarks for teams and OSS.
- Enterprise dashboard — multi-tenant observability for AI ops teams.
Track progress on GitHub Issues , propose a feature on GitHub , or follow the full Roadmap.
Read the docs.
Every surface of Critiqor documented for engineers — concise, complete, runnable.
Getting Started
5-minute walkthrough from install to first diagnosis.
Installation
pip, source, dev install and environment requirements.
CLI Commands
monitor, finalize, dashboard and configuration flags.
Architecture
Pipeline, observer model and artifact lifecycle.
Evidence Collection
Event types, schemas and runtime instrumentation.
Dashboard
Executive summary, runtime timeline and causal views.
Benchmarks (Coming Soon)Soon
How reliability is scored and compared.
API Reference
Python API, plugin interface and extension points.
FAQ
Common questions about observation, privacy and frameworks.
Stay in the loop.
Docs, source, plugins and the conversation around runtime intelligence.
Questions, answered.
Everything developers ask before adopting Critiqor.