Encyclopedia
AI phone agent
What an AI phone agent is, how the speech-to-speech pipeline works, and what it can handle on inbound and outbound calls.
AI phone agent is an AI system that answers or places phone calls, understands the caller through speech recognition, responds with a synthetic voice, and can complete tasks such as answering questions, booking appointments or taking callback requests.
How an AI phone agent works
A phone call is real-time audio, so an AI phone agent runs a low-latency loop of three components:
- Speech-to-text (STT) transcribes the caller's speech into text as they talk.
- A large language model reads the text, decides what to do (including looking up information or calling a system), and produces a reply.
- Text-to-speech (TTS) turns the reply into a natural-sounding voice that is streamed back to the caller.
The whole loop has to stay fast enough to feel like a normal conversation, typically under a couple of seconds per turn, otherwise the caller talks over the agent.
What it handles on a call
- Answering questions from the business's knowledge base.
- Booking or changing an appointment when connected to a calendar.
- Taking a callback request when a human is a better fit.
- Transferring to a team member, with a short summary of the call.
- Switching language when the caller speaks another one.
Inbound and outbound
An inbound agent answers calls to the business, acting as an AI receptionist. An outbound agent places calls, for example to follow up on a list of contacts within set calling hours and with the required consent. Both use the same speech-to-speech loop.
From reference to practice
Turn what you just read into a working AI agent on your own customer contact. Start free, no credit card needed.