← All organizations

LLM evaluation and AI quality control

Root Signals

Root Signals, now operating as Scorable, builds evaluation and monitoring software for teams developing AI applications, chatbots and agents. Its calibrated AI judges score responses for quality and policy adherence, helping teams identify failures in production and test behavioral changes when models or prompts are updated. Developers can create custom evaluators from their own policies and examples, then integrate scoring into applications and release pipelines.

Founded in 2023, the company counts Ari Heljakka among its founders. Its engineering work includes Root Judge, a locally deployable evaluator fine-tuned from Llama-3.3-70B-Instruct. The model detects context-grounded hallucinations in retrieval-augmented generation systems and compares responses using customizable rubrics, with explanations for its scores. Its local deployment option lets organizations evaluate sensitive material without sending it over the public internet.

Scorable serves enterprise AI teams, software vendors building specialized bots and LLM consultants. Its commercial offering includes hosted evaluation software sold through AWS Marketplace under contracts with evaluation allowances. Alongside that service, Root Judge's available weights let developers integrate and customize the model for their own evaluation workflows, providing a choice between managed software and local model deployment.

scorable.ai

2 talks

Newest first

1 speaker at AIE

Affiliations reflect their AIE appearances, not necessarily current employment.

Start here

  1. Agent Evals: Finally, With The Map

    Start here to understand how single-turn and multi-turn evaluation fit into a structured approach to assessing agents.

    Ari HeljakkaAI Engineer Summit 2025

  2. Will Agent Evaluation via MCP Stabilize Agent Networks?

    Use this demonstration to learn how MCP connects agents to persistent judges, including a Cursor example that optimizes marketing copy.

    Ari HeljakkaAI Engineer World's Fair 2025

Messages from the stage

Evaluate the judgment workflow too

Heljakka frames EvalOps as improving both the agent's operative LLM workflow and the separate workflow that judges it. His evaluation map spans grounding, faithfulness, tool-use validation, and API-specific metrics.

Make policy checks accessible through MCP

A hotel-reservation demonstration adds an MCP-accessible policy evaluator to a Pydantic AI agent, preventing it from recommending a competing hotel.

Affiliations reflect each recorded session, not necessarily current employment.

Company sources · checked 2026-08-28