edit_agent_behavior
edit_agent_behavior
Edits what an AI Agent knows, says, or does. Every edit is staged on a change set — a bundle of changes that is not live until it is promoted.
This is the only way to change Agent behavior through the MCP server. Settings such as test cases and topics are edited with edit_agent_config, which applies immediately.
Call get_improvement_guide first, and describe_entity before each edit — it returns the editable fields for an entity type along with the rules that apply to it.
Example prompts
- “Our return window is 30 days now — update the returns article, test it, and publish it if it looks right.”
- “Change step 2 of the returns Playbook and the supporting article together, then run my regression tests against both.”
- “Try that change set on 10% of new conversations, capped at 500, before I promote it.”
- “What behavior changes are staged right now?”
- “That last change is causing confusion — roll it back.”
How a change reaches production
The same loop every time: draft, test, promote, revert. A sampled rollout to real traffic can slot in between test and promote when extra confidence is needed.
- Draft. Describe the AI Agent behavior you would like to change, or the ideal experience you want to build. The MCP assistant decides which entities to edit or create, stages the edit on a change set, and nothing goes live. Related edits can be added to the same change set so they test and ship together.
- Test. Run simulations against the change set. The simulated conversation uses the staged edits while the live Agent stays untouched.
- Roll out. Apply the change set to a sample of new conversations. The staged variants run against real traffic while the rest of the Agent stays on the live version. See Rolling out to a slice of production.
- Promote. The assistant shows what will change and any warnings. Nothing goes live until this is explicitly confirmed.
- Revert. Ask for a revert of a promoted change set. This produces a new testing change set whose edits invert the source. That change set then follows the same loop as any other before anything goes live.
A promoted change set can be reverted for up to 90 days after it was promoted. After that it is automatically deleted and can no longer be reverted.
Testing is strongly recommended but not required — a change set can be promoted without it.
Parameters
Each entry in changes is a created, modified, or deleted change. modified is a partial update — fields that are not supplied keep their live values.
Operations
What you can change
Greetings, handoffs, processes, and CSAT surveys are not covered and continue to be edited in the Ada dashboard.
Rules that apply to each type
These are enforced when an edit is staged, so an edit that breaks one is rejected before it reaches a change set. describe_entity returns the rules for the type requested.
Knowledge articles — only articles created in Ada are editable. Articles imported from another knowledge source, such as Zendesk, Salesforce, or a web import, are read-only and must be edited in the system they came from.
An article can be filed under a specific knowledge source with knowledge_source_id, on both created and modified changes. Find IDs with list_entities using entity_type='knowledge_sources', or create a source with edit_agent_config. Omit the field on created to file the article under Created in Ada; omit it on modified to leave the article where it is. Only a source that holds nothing but Ada-authored articles is accepted. Staging refuses the edit if the target source is being deleted, is managed by an integration such as Zendesk or Salesforce, is a website import, or already holds articles from another system. The staging response names the reason.
Promotion checks each filed article’s source again. If that source is being deleted, no longer exists, or has become a website import, the promote fails and names the article. Remove that change from the change set. Or re-stage it into another source. Then promote again. A revert change set is exempt and restores each article’s original source as-is.
Articles filed into a custom source keep Edit and Delete in the Ada dashboard, and a dashboard edit keeps them in that source.
Coaching — coaching is always anchored to a single Agent turn in a conversation. Take the conversation and event identifiers from a get_conversation response where the entry is marked as coachable. The anchor and the coaching type cannot be changed afterward — to change the type, delete the coaching and create it again. Coaching types that act on a specific target, such as an API tool or a Playbook, require that target to be identified. Coaching copied from another Agent is read-only.
Playbooks — only Playbooks built from sections and steps are editable; older Playbooks Classic entries are not. If a live Playbook contains send-step attachments, its sections cannot be edited here, though its name, description, and general instructions still can. Every staged Playbook is validated the same way the dashboard validates a Playbook on save, so an incomplete Playbook cannot be staged.
API tools — the name, description, enabled state, and direct-use setting can be changed. The request itself, along with the input and output schemas, forms the contract with any Playbook that uses the API tool, so those are not editable — changing them in place would break those Playbooks silently. Create a new API tool instead.
Code tools — a code tool can be created, updated, and deleted through the change-set flow, the same draft, test, promote, revert loop as any other entity. Its name, description, and code can be changed. Its inputs, outputs, and environment variables are set at creation and fixed afterward, since they form the contract a Playbook step relies on — changing them means recreating the tool, which mints a new one with a new identifier. A promoted delete stays revertible for 90 days.
MCP tools. An mcp_tool is a tool provided by a connected MCP server. Its name, description, and input schema come from the server and are read-only. Only enabled, requires_user_auth, direct_use, and availability_rules can be changed, and only on modify. Creation and deletion of an MCP tool happen on the server side, not through change sets. requires_user_auth is valid only on an OAuth connection. The tool-level enabled flag does not override its connection’s kill switch, so enabling a tool on a disabled connection has no effect until the connection is re-enabled.
MCP server connections. An mcp_server_connection is the connected MCP server that provides mcp_tool entries. Only enabled and name are editable, and only on modify. Creation, deletion, credential setup, and refreshing the connection’s tool list all happen in the Ada dashboard. enabled is the connection-level kill switch: when false, none of the connection’s tools are offered or run, regardless of each tool’s own enabled flag. A rename that collides with another connection’s name is rejected at stage time.
Both mcp_tool and mcp_server_connection require the MCP tools entitlement on the Agent. On an Agent without it, the stage is rejected with a message that MCP tools cannot be edited through a change set.
Availability rules are not set through change sets. They are configured in the Ada dashboard, and are preserved when a change set is promoted or reverted.
Limits
Reaching a limit returns an explanation. Promote or delete an existing change set to free capacity.
Approval before anything goes live
Promoting or discarding a change set always pauses for approval. The assistant shows what will change and any warnings, then waits for explicit confirmation before applying it. This cannot be skipped, even by asking the assistant to publish without confirming.
revert does not need this approval, because it does not change live behavior on its own. It only produces a new testing change set. That revert change set is then subject to the same promotion approval as any other change set before it goes live.
If the live Agent changes again between the summary and the approval, the assistant checks again and shows an updated summary rather than applying something that was never reviewed.
When two edits clash
More than one person can work on the same Agent, and the live Agent can change while a change set is being staged. Before a change set goes live it is checked against the live Agent, and any conflict is surfaced as a warning. Conflicts are never merged automatically and never applied silently.
Edits to different fields of the same entity both survive. Edits to the same field are resolved in favor of whichever change set is promoted last.
Warnings appear in the approval summary and do not block promotion:
When warned, there are three ways forward: promote anyway and let the staged version win, ask the assistant to rebuild the edit on top of the current live version and then promote, or discard the edit and keep what is live. The change set is preserved in all three cases.
Rolling out to a slice of production
A change set can serve a sample of new conversations before it is promoted. Its staged variants run against real traffic while the rest of the Agent stays on the live version.
Start sampling with set_rollout on a testing change set. Two settings are required:
percentage: the share of new conversations to sample, from 1 to 100. Each new conversation is drawn independently at this rate.max_conversations: the hard cap on how many conversations the change set ever applies to. The rollout completes the moment this cap is reached.
Percentages across all active rollouts on the same Agent must sum to at most 100. A request that would push the total over is rejected.
Only new conversations are sampled. A conversation already in progress keeps its current variants.
Stop sampling with stop_rollout. No further conversations are sampled in, and the percentage frees back to the 100% cap. Conversations already pinned to the change set keep serving its variants until they end.
To resume sampling on the same change set, call set_rollout again. The applied count resets to 0.
A rollout does not promote the change set. It stays in the testing state, and promotion is a separate step once the rollout has built confidence.
Measuring a rollout against a baseline
Conversations attributed to a change set can be scoped in metric queries using the CHANGESETID filter on get_ada_metric. Pass the change set’s id to see the rollout cohort. Pass value=['baseline'] with IS to scope to conversations not on any change set. That baseline cohort is the control for A/B comparison against the rollout. The full list of filters is on get_available_filters.
Promoting and reverting
Promotion is all-or-nothing. Every staged edit is applied together, and if any part fails the whole change set is rolled back. One change set is promoted at a time per Agent, so a promotion attempted while another is in progress is rejected and can be retried.
Only a promoted change set can be reverted. revert produces a new testing change set whose edits invert the promoted source, restoring each entity to its state before the source was promoted. Nothing is applied to live at this point.
That revert change set then follows the same loop as any other:
- Inspect and adjust it.
- Run a test on the proposed restore.
- Optionally roll it out to a slice of production.
- Promote it.
Any edit made to those entities after the source’s promotion is overwritten once the revert change set is promoted.
Only one testing revert of a given source can exist at a time. Promote or delete the existing revert change set before starting another for the same source.
Reverting and deleting are different. Reverting produces a new testing change set that inverts a promoted source. Promoting that revert change set is what returns live behavior to the previous state. Deleting only discards a testing change set and touches nothing live. To undo a live change, use revert.
Change sets are promoted within a single Agent. A change set cannot be moved between instances.
Retention
Change sets have a 90-day lifetime. A testing change set is automatically deleted 90 days after it is created. A promoted change set is automatically deleted 90 days after it is promoted, so it stays revertible only within that window.
Where to see results
Change sets are managed through the MCP server. The Ada dashboard adds a Change sets page at Config > AI AGENT > Change sets where you can browse and inspect every change set on your Agent.
Simulations shows each test case’s latest result across live and change set runs. Retrieve their results with get_test_runs, and use list_agent_changesets to see what is staged, promoted, or reverted from the MCP server.