Skip to navigation

Custom Metric verdicts

These endpoints are in limited availability. If they are not enabled for your AI Agent, the response is a 403.

Use these endpoints to copy the verdicts of your Custom Metrics into your own data warehouse. You load them into the same warehouse as the conversations and messages that you export.

What the export contains

The Data Export API (v2) has three endpoints for Custom Metric verdicts.

EndpointWhat it returns
GET /api/v2/export/custom-metric-verdictsThe verdict feed. Each record is one state of one conversation and one metric.
GET /api/v2/export/custom-metricsThe list of your Custom Metrics, with the name, the direction, and the state of each metric.
GET /api/v2/export/custom-metric-verdicts/countsThe number of conversations and metrics in a time window, by metric and verdict. Use it to check your copy.

The endpoints use the same host and the same API key as the conversations and messages endpoints. For how to send the key, see Authentication. Each response has the records in data and the response details in meta. The conversation record does not change.

Your copy has one row for each conversation and metric, with the latest verdict. Load the verdicts into their own table, for example custom_metric_verdicts. The key of a row is (conversation_id, metric_id). Join the table to your conversations table on conversation_id = _id. Join it to the metric list on metric_id.

Build the polling job

The first run and every later run use the same steps. A crawl is the set of pages from one first call to the page where next_page_uri is null.

  1. Call the verdict feed. On the first run, send your start date in updated_since, for example GET /api/v2/export/custom-metric-verdicts?updated_since=2026-01-01T00:00:00Z. On every later run, call the next_poll_uri that you saved.
  2. Follow meta.next_page_uri until it is null. It is a full URL. Call it as it is.
  3. Load the records of each page with the load rule.
  4. When next_page_uri is null, save meta.next_poll_uri. The next run calls it.
  5. Call GET /api/v2/export/custom-metrics and update your metric table. For the steps, see Keep the metric list current.

A page can hold fewer records than page_size, or none, and still have a next_page_uri. A short or empty page is not the last page. Only null ends the crawl. Neither URL expires, so you can stop at any next_page_uri and continue from it later.

Getting started has a sample in JavaScript and Python that follows next_page_uri through the messages endpoint. The verdict feed pages in the same way.

Load each record

Use this load rule for every record that you get from the verdict feed.

  • Upsert on (conversation_id, metric_id).
  • Keep the record with the greatest record_last_updated. Insert a new key. Overwrite a row when the new record_last_updated is greater than or equal to the one that you hold.
  • Load a record with is_removed: true in the same way. Keep it as a row. For more information, see Keep removed records.

The order of loading does not matter. You can get the same key more than once, also in one crawl or on one page. Two records with the same key and the same record_last_updated are the same state.

For one key, a newer state always has a greater record_last_updated. So the row with the greatest value is the current verdict, in whatever order you load the pages. If you load a page with one MERGE statement, first reduce the page to one record for each key, the record with the greatest record_last_updated.

How often to poll

Poll once an hour as a default. Any interval from 15 minutes to 24 hours gives the same rows. A longer gap is also safe, because nothing expires. A verdict appears 1 to 3 minutes after Ada can read it.

Call the metric list on every run.

Example

This call is a later run of a job. The job calls the next_poll_uri that it saved. The IDs and the text are examples.

GET https://example.ada.support/api/v2/export/custom-metric-verdicts?updated_since=2026-11-17T09:00:00.000Z

{
"data": [
{
"conversation_id": "6739a1f2c4e5d60012ab34cd",
"metric_id": "671f0c2e9b1d4a0011223344",
"is_removed": false,
"verdict": "fail",
"reasoning": "The AI Agent asked for the order number again after the end user gave it.",
"metric_version": 3,
"judge_version": 2,
"evaluated_at": "2026-11-17T09:04:00Z",
"scored_at": "2026-11-17T09:14:07Z",
"is_final": false,
"record_last_updated": "2026-11-17T09:15:02.118Z"
},
{
"conversation_id": "6738ffa0c4e5d60012ab1200",
"metric_id": "671f0c2e9b1d4a0011225566",
"is_removed": true,
"verdict": null,
"reasoning": null,
"metric_version": null,
"judge_version": null,
"evaluated_at": null,
"scored_at": null,
"is_final": null,
"record_last_updated": "2026-11-17T09:30:01.940Z"
}
],
"meta": {
"next_page_uri": null,
"updated_to": "2026-11-17T10:58:02.004Z",
"next_poll_uri": "https://example.ada.support/api/v2/export/custom-metric-verdicts?updated_since=2026-11-17T10%3A58%3A02.004Z"
}
}

The job loads both records by key. The first record is an early fail, because the conversation was open when the judge read it. The second record says that Ada no longer returns a verdict for that conversation and metric. The job keeps that row and leaves it out of reports.

next_page_uri is null, so this page ends the crawl. The job saves next_poll_uri and calls it on the next run. The conversation that failed ends about a day later. Then a second record for the same key arrives with is_final: true and a greater record_last_updated, and it replaces the first one.

The same run calls the metric list.

GET https://example.ada.support/api/v2/export/custom-metrics

{
"data": [
{
"metric_id": "671f0c2e9b1d4a0011223344",
"name": "No repeated questions",
"direction": "higher_is_better",
"state": "active"
}
],
"meta": {
"next_page_uri": null
}
}

The metric has the direction higher_is_better, so the fail above counts against the metric. On a metric with the direction higher_is_worse, a pass counts against the metric. For more information, see Verdict values and direction.

Join the verdicts to your other tables

Join the verdict table to your other tables with these conditions:

  • custom_metric_verdicts.conversation_id = conversations._id
  • custom_metric_verdicts.metric_id = custom_metrics.metric_id

Use a LEFT JOIN from the verdict table. A verdict can point to a conversation that you do not hold. The conversation can be older than the history that you loaded, or older than the 12-month conversation history. Or your conversations job has not reached it yet.

Do not select verdicts by a conversation date. A new verdict does not change the conversation’s date_updated. Get new verdicts from the verdict feed.

Load your history

The first run loads every record from your start date. Verdicts have no age limit. The 12-month limit in the overview is for conversations and messages.

For the time before Ada started to record the export, you get one record for each key, the current verdict. Its record_last_updated is the time when the judge wrote the verdict. After that time, you get every state that the export saw. A verdict that was written and replaced between two runs of the export, about a minute apart, does not appear.

You can stop and resume at any next_page_uri. You can also load slices of history side by side. Cut the history into slices with updated_since and updated_to. Each slice ends where the next one starts. The slices can run at the same time and in any order, because the load rule does not depend on the order. Keep the next_poll_uri of the newest slice only.

To poll new verdicts during a long load, do these steps:

  1. Make one call with updated_since set to your start date and page_size=100. Take T0 from its meta.updated_to. Do not take T0 from your own clock.
  2. Load the history with updated_to=T0.
  3. Start the hourly poll at updated_since=T0.

If the meta.updated_to of a slice is earlier than the updated_to that you sent, the slice ended early. Start the next slice, or the poll, at that meta.updated_to.

Keep the metric list current

Call GET /api/v2/export/custom-metrics on every run. A 200 always holds the full list. The list has no pages, and it can be empty.

After a 200, upsert every metric on metric_id. A metric that you hold and that is not in the list was deleted. Mark it as deleted, and hide its verdict rows.

Change in AdaWhat your job sees
A metric is renamed, or its direction changes.Nothing in the verdict feed. The next metric list has the new value.
The text of a metric is edited.Ada does not score past conversations again. Later verdicts have a higher metric_version.
A metric is deactivated.The list shows state: draft. The dashboard shows the metric as Inactive. Its past verdicts stay. No new verdicts arrive.
A metric is deleted.The metric is not in the list. The verdict feed, the counts, and the reconcile pull stop returning its verdicts. Hide its rows, as the dashboard does.

The list gives no time for a rename. It does not say which metric_version is current. If you need either one, keep your own history table.

How verdicts change

A conversation gets a first verdict while it is open, with is_final: false. It gets a new record when it ends. A chat ends about 1 day after the last message, and an email conversation about 3 days after it. A conversation in a live agent handoff ends when the handoff ends. Past conversations can get a verdict months later, when a metric is backfilled.

Each change arrives as one more record for the same key on a later poll. This is true for a late verdict, a changed verdict, a backfill verdict, a removed verdict, and a verdict of a deleted conversation. A verdict that changes to No verdict arrives as a record with verdict: "unjudged".

Keep removed records

A record with is_removed: true means that Ada no longer returns a verdict for this conversation and metric. Ada sends it for one of two causes:

  • Ada took the verdict back. For example, a metric scoped to an Intent gave a verdict while the conversation was open. When the conversation ended, it did not have that Intent.
  • The conversation’s data was deleted on a deletion request.

The record does not say which of the two caused it, so do not use it to start a deletion in your own warehouse. A deletion by a data retention period does not send this record.

Keep the row with its record_last_updated, and leave it out of reports. Do not delete the row. If you delete it, an older page that you load later brings the old verdict back.

In a removed record, every field is null except conversation_id, metric_id, is_removed, and record_last_updated. When you load it, your row loses the verdict and the reason text.

Final and early verdicts

is_final says what Ada recorded at the moment when the judge read the conversation.

is_finalMeaning
trueAda recorded that the judge read the conversation after it ended.
falseAda recorded that the judge read the conversation while it was open.
nullAda does not know if the conversation had ended when the judge read it.

Every verdict that was scored before Ada started to record this fact is null. So are a few later verdicts and every removed record. The value of a record does not change. A later verdict arrives as a new record.

  • For the verdicts that Ada confirms are final, keep the rows where is_final is true. This leaves out the verdicts from before Ada recorded the fact.
  • To drop only the verdicts that Ada knows are early, drop the rows where is_final is false.

A row can stay false for good. This happens when the metric was set to draft or stopped applying before the conversation ended. It also happens when the conversation reopened after it ended, for example a voice call that continues by SMS. A true verdict can still be replaced by a backfill or a re-score.

Verdict values and direction

The verdict field has one of these values. For what each verdict means, see Verdicts and statuses.

verdictName in AdaMeaningCounts toward the pass rate
passPassThe conversation meets the pass criteria.Yes
failFailThe conversation meets the fail criteria.Yes
not_applicableNot applicable, shown as N/AThe conversation matches When this metric does not apply.No
undecidedUndecidedThe metric applies, but its criteria and Scoring guidance don’t decide the conversation.No
unjudgedNo verdictThe judge couldn’t score the conversation. reasoning is null.No

Ada can add new verdict values. Leave a value that you do not know out of your pass rate.

The direction of a metric says which result the metric wants. higher_is_better is Higher pass rate is better. higher_is_worse is Lower pass rate is better. On a metric with direction higher_is_worse, a pass means that the unwanted thing happened. Read direction from the metric list before you count passes as good.

When late verdicts and removals arrive

Ada adds a verdict to the feed 1 to 3 minutes after its database can read it. For one conversation and metric, a newer record always has a greater record_last_updated. Keep the greatest one, and your row is the current verdict.

If Ada’s database makes a verdict readable late, Ada adds the verdict when Ada sees it. You do nothing.

The verdict is readable late byIt arrives in the feed within
Up to 29 hours15 minutes
Up to 7 days1 day
Up to 39 days8 days

When Ada takes a verdict back, a record with is_removed: true arrives. In about 95% of cases, it arrives within 15 minutes. It also arrives within these times:

Ada stored the verdictThe removed record arrives within
Up to 7 days before1 day
Up to 39 days before8 days

When a conversation’s data is deleted on a deletion request, a record with is_removed: true arrives for each of its verdicts. It normally arrives within a few minutes. If the first check misses the deletion, a full check that Ada completes at least once a day finds it. The record then arrives within 2 days.

Ada does not find, by itself, a verdict that becomes readable or is taken back more than 39 days after Ada stored it. To repair such a case, contact your Ada team. Ada can run the same check by hand for any period, and the correction then arrives in the feed like any other record.

When Ada takes a verdict back, the dashboard stops counting it at once. Your copy drops it when the removed record arrives.

Check your copy with counts

The counts endpoint counts the keys whose current record has a record_last_updated in a window, from updated_since up to but not including updated_to. It groups them by metric_id, is_removed, and verdict. Each count is a number of keys, the same unit as one row of your table. A group with no keys is not in the response.

  • Both parameters are required. The window is at most 24 hours long.
  • The window must end at or before the meta.updated_to of your last finished crawl. Then it also ends at or before the export bound, the time up to which Ada has added records. A later end gives a 400.
  • The counts are as of the moment of the call. meta.as_of gives that time. No parameter asks for another time.
  • A window with records of more than 250,000 conversations gives a 503 on every call. Count shorter windows that together cover it, and add their counts.

Check once a day, right after a crawl. Check the UTC day that ended 2 days ago. You can check a window again at any time.

GET https://example.ada.support/api/v2/export/custom-metric-verdicts/counts?updated_since=2026-11-13T00:00:00Z&updated_to=2026-11-14T00:00:00Z

{
"data": [
{"metric_id": "671f0c2e9b1d4a0011223344", "is_removed": false, "verdict": "pass", "count": 4120},
{"metric_id": "671f0c2e9b1d4a0011223344", "is_removed": false, "verdict": "fail", "count": 311},
{"metric_id": "671f0c2e9b1d4a0011223344", "is_removed": true, "verdict": null, "count": 2}
],
"meta": {
"as_of": "2026-11-17T10:59:12.000Z"
}
}

Count the same window in your copy. Use only the metrics that are in the metric list.

SELECT metric_id, is_removed, verdict, count(*)
FROM custom_metric_verdicts
WHERE record_last_updated >= '2026-11-13T00:00:00Z'
AND record_last_updated < '2026-11-14T00:00:00Z'
AND metric_id IN (/* the metric_id values in the metric list */)
GROUP BY 1, 2, 3
  • If any group differs, repair the window with reconcile. You do not need a tolerance.
  • If every group is equal, no difference was found. This is strong evidence, but it is not proof. One missing row and one extra row in the same group cancel out. The reconcile pull is the proof.

A key can change between your crawl and the count call. Then your count is higher by that key until your next crawl.

The count includes a key whose verdict Ada took back, until Ada adds the removed record. In that time, the feed and the reconcile pull do not send the old verdict of that key.

  • A job that polled all along holds the old verdict, so its counts match.
  • A job that loads the window for the first time in that time is lower by that key. A reconcile does not close the difference. The difference ends when the removed record arrives. That is within 15 minutes in most cases, and within 8 days for a verdict that was stored up to 39 days before.

When a conversation’s data is deleted, the count shows each of its keys as removed at once, before Ada adds the removed record. The count keeps the key in the window that holds the record_last_updated of its last verdict, in the group with is_removed: true. In that time, the feed and the reconcile pull do not send the old verdict of that key.

  • A job that polled all along holds the old verdict. Its count of that verdict is higher by that key, and its count of removed keys is lower by that key.
  • A job that loads the window for the first time in that time has no row for that key. Its count of removed keys is lower by that key.

A reconcile does not close these differences. They end when the removed record arrives, normally within a few minutes and within 2 days. The key then moves to the window of the removed record, in the counts and in your copy.

So a difference in a window can last up to 8 days. A difference can remain after a reconcile, after one more crawl, and 8 days after you first found it. Then it is a fault on Ada’s side. Contact your Ada team with the window and the groups that differ, with both counts.

Repair a window with reconcile

To repair the window from A up to but not including B, do these steps:

  1. Call GET /api/v2/export/custom-metric-verdicts?updated_since=A&updated_to=B&reconcile=true.
  2. Follow next_page_uri until it is null.
  3. Load the records with the load rule.

A reconcile pull returns the current record of every key that has a record in the window. A key whose state changed later comes back with its later state and its later record_last_updated. So a record can have a record_last_updated after B.

  • A reconcile has no delete step. Every row that you hold with a time in the window came from a record in the window. So the pull returns its key with its current state.
  • You do not need a crawl before or after it. You can repair windows in any order. A reconcile can run while your poll runs.
  • A reconcile pull gives no next_poll_uri, so it does not move your checkpoint.
  • A reconcile works for any window of the exported history, however old.
  • Both updated_since and updated_to are required. The window is at most 24 hours long. It must end at or before the export bound, or the response is a 400.

Match the dashboard pass rate

The dashboard does not put a verdict on a day by a verdict time. It counts the latest verdict of each conversation, final or not, on the day of the conversation’s first engaged message inside the chosen range. It uses the time zone of the dashboard.

To get close to the dashboard’s pass rate in your warehouse, do these steps:

  1. Join each verdict to its conversation.
  2. Keep the conversations where is_engaged is true.
  3. Leave out the rows where is_removed is true.
  4. Put each verdict on the day of the conversation’s date_created.
  5. Compute the pass rate as pass / (pass + fail).

Do not use evaluated_at or scored_at for the day.

The result is close to the dashboard, but it is not exact. The size of the difference is not measured. These are the known causes:

  • The first engaged message is on a later day than date_created.
  • You do not hold the conversation of a verdict.
  • The conversation’s data was deleted. The dashboard still counts its verdict, and the export marks the verdict as removed.

Reference

Verdict record

FieldTypeCan be nullMeaning
conversation_idstringNoThe _id of the conversation record, 24 hexadecimal characters.
metric_idstringNoThe metric_id of the metric in the metric list.
is_removedbooleanNotrue when Ada no longer returns a verdict for this conversation and metric. See Keep removed records.
verdictstringWhen is_removed is truepass, fail, not_applicable, undecided, or unjudged. See Verdict values and direction.
reasoningstringYesThe reason that the judge gave, with private data hidden as the Ada MCP server hides it. null for unjudged.
metric_versionintegerYesThe version of the metric text that gave this verdict. null means that the verdict is from before Ada recorded versions. It does not mean version 1.
judge_versionintegerYesThe version of Ada’s judge. null on verdicts from the older judge.
evaluated_atstringWhen is_removed is trueThe time of the conversation snapshot that the judge read. For a verdict given while the conversation was open, it is the time of the end user’s last message.
scored_atstringWhen is_removed is trueThe time when the judge wrote this verdict.
is_finalbooleanYesWhether Ada recorded that the conversation had ended when the judge read it. See Final and early verdicts.
record_last_updatedstringNoThe time when Ada added this state of the conversation and metric to the export. updated_since and updated_to select on it. For one key, a newer state always has a greater value.

The meta object of the verdict feed has these fields.

FieldTypeMeaning
next_page_uristring or nullThe full URL of the next page of this crawl. Only null ends the crawl. It does not expire.
updated_tostringThe end of the window of this crawl. It is the same on every page of the crawl.
next_poll_uristring or nullThe first page of the next crawl. It is set only on the last page of a crawl without reconcile. It is the same endpoint with updated_since set to updated_to, plus the page_size of this crawl if you sent one. It does not expire.

Metric record

FieldTypeCan be nullMeaning
metric_idstringNoThe ID of the metric.
namestringNoThe current name of the metric.
directionstringNohigher_is_better or higher_is_worse. See Verdict values and direction.
statestringNoactive or draft. The dashboard shows a draft metric as Inactive.

The meta object of the metric list has one field, next_page_uri. It is always null.

Counts record

FieldTypeCan be nullMeaning
metric_idstringNoThe ID of the metric.
is_removedbooleanNoWhether the current records of the group are removed records.
verdictstringWhen is_removed is trueThe verdict of the group.
countintegerNoThe number of keys in the group. It is at least 1.

The meta object of the counts has one field, as_of. It is the database time when Ada read the counts.

Parameters of the verdict feed

ParameterRule
updated_sinceRequired, unless you send cursor. The feed returns records with a record_last_updated at or after this time.
updated_toOptional. Without reconcile, the feed returns records with a record_last_updated before this time, and a later time is lowered to the export bound. For reconcile, see Repair a window with reconcile. The default is the export bound. meta.updated_to gives the time that the feed used. Without reconcile, the window has no maximum length.
reconcileOptional. The default is false. With true, the feed returns the current record of every key that has a record in the window. See Repair a window with reconcile.
page_sizeOptional. From 100 to 10,000 records. The default is 10,000.
cursorOnly next_page_uri sets it. You cannot send it with another parameter.

The export bound is the time up to which Ada has added records to the export. A crawl never returns a record after it. The meta.updated_to of a crawl is at or before the export bound.

  • Without reconcile, every record has a record_last_updated at or after updated_since and before meta.updated_to.
  • If updated_since is later than the export bound, the response is a 200 with an empty data. meta.updated_to is equal to updated_since, and next_poll_uri has the same updated_since. Your checkpoint never moves back.

Times

Every time in a response is in UTC and ends with Z, never with +00:00.

  • record_last_updated, meta.updated_to, and meta.as_of have milliseconds, for example 2026-11-17T09:15:02.118Z.
  • evaluated_at and scored_at have whole seconds, for example 2026-11-17T09:04:00Z.

The conversation record also has a field named record_last_updated. That field uses microseconds and +00:00, for example 2026-11-17T09:15:02.118000+00:00. Do not compare the two fields with each other.

Send times in UTC with Z, for example 2026-11-17T09:00:00Z. A time with no Z and no offset gives a 400. If you send an offset, encode the + as %2B. An unencoded + in a query string becomes a space.

Status codes

The three endpoints use these status codes. The metric list has no parameters, so it does not return a 400.

StatusMeaningWhat your job does
200The complete answer. A 200 is never partial.Load it.
400A parameter or the cursor is not valid. For counts and reconcile, the window can also end after the export bound.Stop. Fix the call.
401The API key is not valid.Stop. Check the API key.
403The export of Custom Metric verdicts is not turned on for this AI Agent.Stop. Contact your Ada team.
404The Agent does not have Custom Metrics.Stop.
429The call is over the rate limit. The Retry-After header gives a time in seconds.Wait that many seconds. Then send the same URL.
500An unexpected error on Ada’s side.Wait. Then send the same URL.
503A store that the answer needs could not be read in time. For counts, the window can also have records of more than 250,000 conversations.Wait 5 seconds, then send the same URL. Double the wait after each 503, up to 5 minutes. If the 503 continues for an hour, contact your Ada team. Nothing expires in that time. If the message says that the window has records of more than 250,000 conversations, do not send the same URL again. Count shorter windows that together cover it, and add their counts.

Set the timeout of your HTTP client to at least 30 seconds.

Rate limits

Each of the three endpoints has its own limit: 10 requests a second, 100 a minute, and 15,000 a day. Your conversations and messages jobs do not share it. A 429 carries Retry-After. Wait that many seconds and send the same URL. Responses carry the X-RateLimit-* headers. For more information, see Limits.

Scope and privacy

The export leaves out these items:

  • Simulated conversations.
  • Metrics pinned to a change set.
  • The quote that the judge gives with a verdict.

Test-user conversations are included, as in the dashboard’s pass rates. To leave them out, filter on the conversation’s is_test_user.

The reasoning text has private data hidden, as in the Ada MCP server. If a conversation’s data was deleted, the feed does not return its verdicts, and the counts show its keys as removed. This is true before the removed record arrives too. The removed record has no text.

Limitations

These limits apply to the export of Custom Metric verdicts.

  • Ada finds a verdict that is readable late, or taken back, for up to 39 days after Ada stored it. For a later case, contact your Ada team. See When late verdicts and removals arrive.
  • is_final is null for every verdict that was scored before Ada started to record it.
  • A removed record arrives after the dashboard drops the verdict. That is within 15 minutes in about 95% of cases, and later for older verdicts.
  • Equal counts are strong evidence, not proof. The reconcile pull is the proof.
  • A pass rate from your copy is close to the dashboard’s pass rate, but not exact.
  • A verdict that was written and replaced between two runs of the export does not appear.
  • The metric list has no history of renames, and it does not say which metric_version is current.

What can change

Ada can add fields and new verdict values. Ada does not rename or remove a field. Load the records in a way that ignores a field that you do not know.