AI Agent reads images sent over Email
Your AI Agent now reads images that end users attach to emails and uses them to resolve inquiries. A screenshot of an error, a photo of a damaged product, or an order confirmation becomes part of the conversation. To turn it on, set the inbox’s attachment handling to AI Agent replies using the attachment.
What’s new
The Agent reads images only on inboxes set to AI Agent replies using the attachment.
- Image descriptions: the Agent receives a text description of each image, including any visible text such as order numbers and error messages. Up to 3 images per email are read. See Images for formats and limits.
- Attachment handling controls it: set the inbox to AI Agent replies using the attachment to have the Agent read images. On the default Hand off to a human, every email with an attachment is handed off, images included. Inboxes that discard attachments never read images. See Attachment handling.
- Sensitive images are withheld: the Agent detects images that show a payment card, a government identity document, or another credential. For those, it receives only the document type and none of the text on it. Detection is automatic and best-effort, and your redaction rules apply to every description. The Agent tells the end user it could not process the image and offers a human agent where needed.
- Redactions apply: your redaction rules run on image descriptions the same way they run on typed text.
Send your Voice AI Agent's texts through your own SMS provider
Voice AI Agents can now send their in-call text messages through your own SMS provider and phone number. This needs a developer. They create a Conversations API channel with the new sms modality and run a relay that receives each text message as a v1.conversation.message webhook and sends it with your provider.
Create the channel with POST /v2/channels. Set modality to sms. Then turn on the Use your own SMS channel toggle in Config > CHANNELS > Voice > Configuration. Until the toggle is on, the AI Agent sends its text messages from the Twilio number configured for your AI Agent.
Ada tags each webhook event from this channel with the SMS channel’s ID, so you can scope a webhook endpoint to the channel. The event carries no phone number. Read the caller’s number from the end user that data.end_user_id names. Ada records no delivery for a text message on this channel. Report delivery in your own relay. For setup steps, payload shapes, and limits, see Relaying voice texts through your own SMS provider.
The caller can reply to a text message during the call. Your relay posts the reply to POST /v2/conversations/{conversation_id}/sms/ as the end user. The AI Agent answers by voice on the call. See Accept the caller’s reply.
AI Agent reads HEIC and HEIF images
Your AI Agent now reads HEIC and HEIF images, the default photo format on iPhone. An end user who emails a photo straight from their camera roll no longer needs to convert it first.
What’s new
HEIC and HEIF join PNG, JPEG, and GIF as image formats the Agent reads on Email.
- Converted on receipt: each HEIC or HEIF image the Agent reads is converted to JPEG when it arrives. The JPEG is what the Agent reads and what human agents receive on handoff.
- Large photos are scaled: images wider or taller than 4096 pixels are scaled down during conversion. Images over 50 megapixels are not read.
- Same rules as other images: the inbox’s attachment handling setting, the 3-image limit, sensitive-image detection, and your redaction rules apply unchanged. See Images.