Custom evals, built with your team
Our forward-deployed engineers embed with your team and build your evals on our platform, then add new tests every week as your product changes.
How we work
Five layers,
each built from your workload
A Newmeasure eval is a working copy of the job your agent does, built by our engineers on our own platform.
We learn the workload your agent automates. We read real requests and traces with your team and map the jobs your customers pay for.
Endpoints per app
- CRM58
- Billing44
- Email38
- Chat26
- Docs22
The initial state: accounts, records, history and the mess real companies carry.
Months of history, thousands of records, real-looking people
OldestActivityToday
Stale accounts and duplicates, left in on purpose
What customers actually ask for, written from real requests and traces.
TaskOpen
“Refund the duplicate charge and let the customer know in chat.”
Each task written from a real request
Checks that the agent did the right thing, read from the end state of the software.
- refund_issued (order)Pass
- no_duplicate_refund ()Pass
- customer_notified (chat)Fail
Read from the end state of the software
The model and harness under test. Swap either one and run again.
Model
claude-opus-5.5
or gpt-6-astra
Harness
claude-agent-sdk
or acme-harness
Ongoing evals
An eval set that grows with your product
Every week, what went wrong in production becomes new tests and every release has to pass them.
+40 every Monday
Tasks in total
Before you sleep
Release score
Platformized services
Want us to build it? We embed with your team
Our engineers build your evals on Benchmark Router and run them with you.
book a callPrivate evals
A test suite only your team sees.
BuildLive new task every weekDomain benchmarks
Public benchmark with your name on it.
BuildLaunchForward-deployed engineers
Eval engineers in your team, on our platform.
Inside your team for a quarter
