Job Description
- Job Description:
- Production reliability: ensuring HAP (our Hybrid AI Platform) and customer deployments operate dependably under real traffic, real data, and real regulatory constraints
- Agent trust: building the frameworks — technical and operational — that allow enterprise customers to trust autonomous AI agents doing work on their behalf
- A production quality operating system: quality gates, phase transition criteria, incident taxonomy, observability spec across our 6-layer Reference Architecture
- A continuous validation framework for agentic workflows — autonomous evaluation pipelines that catch regression without human intervention
- An agent decision qualification framework: risk-tiered oversight for autonomous agent decisions
- A trust evidence system: the observable signals that enterprise customers use to extend trust to agents operating on their behalf
- Production observability: instrumentation across Ingest, Prepare, Serve, Orchestrate, Monitor, and Optimize layers of the Reference Architecture
- A post-mortem and CAPA system: every production incident produces a root cause, a corrective action, and a new test that prevents recurrence
- Requirements:
- 10+ years in quality, reliability, or production operations for complex distributed systems — with at least some of that time governing AI or ML systems in live production
- Direct implementation experience with AI quality frameworks — you built it, not just led a team that built it
- Familiarity with the agentic AI quality problem: non-deterministic systems, hallucination detection, behavioral drift, autonomous decision governance
- Working knowledge of open-source evaluation and observability frameworks (LangSmith, Arize/Phoenix, RAGAS, PromptFlow, Weights & Biases, or similar) — not just commercial alternatives
- Background in regulated industries (financial services, telecom, healthcare, government) where AI quality failures have real contractual and commercial consequences
- Benefits:
- Startup orientation: comfortable with ambiguity, iterative scope, and a team that moves faster than most people expect
Apply tot his job
Apply To this Job