Evals & Observability
Tools for evaluating, tracing, and monitoring AI systems in development and production.
3 tools
Codex Quota Overlay
Codex Quota Overlay contributors
Independent desktop companion for Codex that displays quota, reset time, and reset credits beside the conversation title on Windows and macOS. It monitors usage; it is not a coding model or an official OpenAI product.
Langfuse
Langfuse
Open-source LLM engineering platform for tracing, evals, prompt management and metrics, with OpenTelemetry-based observability. Self-hostable; the core is MIT-licensed with paid enterprise features.
- Open source
- Local-first
- Self-hostable
Sentinel
Nano Collective
Installable, Nanocoder-driven workflow that runs continuous, configurable security and code audits across the repositories in a GitHub organisation and files findings as issues. Local models are a first-class path.
- Open source
- Local-first
- Privacy-first
- Self-hostable