The Comprehensive Guide to AI Phone Calling

Introduction

Generative AI has enabled a whole new set of technologies to emerge. None are more exciting and relevant than AI phone calls.

For the first time ever, artificial intelligence can replace humans on the phone. Don’t take my word for it - you can send yourself a test call here. The AI phone agent can follow instructions, adapt on the fly, and complete any task you provide. But what’s the catch?

Emerging technology is hard to understand, apply, and scale. Most organizations don’t even know where to get started. Prompting excellent conversational output from an AI is tricky; teaching it to effectively complete a task is hard. Plus, how can you trust that when you put the AI in front of your customers it won’t go off the rails?

During the past twelve months, our team at Bland AI has heard and answered many of these questions. We’ve consistently built custom AI phone calling integrations for large enterprises across the United States and the rest of the world and we understand the best practices for piloting, implementing, and scaling AI phone operations.

This guide is the accumulation of our AI phone calling expertise. We’ll include everything you need to know about AI phone calls, from how the underlying technology works, to the best use cases for AI phone calls, to a clear explanation of how your organization can implement them.

You’re about to become an AI phone-calling expert. Read on!

How AI phone calls work

Here’s the most important thing to understand: an AI phone call is not a traditional robocall. It’s dynamic.

During AI phone calls, the AI phone agent listens to the speaker and then responds, just like a human. This means AI phone agents sound and feel human. Because they’re dynamic, they can hold conversations about any topic, sometimes even better than a human.

We use three underlying models to make these conversations happen:

1. A transcription model that listens to audio and converts it to text.
2. A language model that decides how to reply
3. A text-to-speech model that converts the language model’s response into human voice
   

Because most of these models are publicly available, creating a rudimentary AI phone calling system is somewhat easy. We even detail exactly how to do it in this guide.

The issue most people face after building an AI phone calling system, however, is that the latency is way too high. Meaning the AI takes five seconds or more to respond to what the speaker says. Moreover, the AI fails to process interruptions and starts speaking over itself.

To solve both problems, AI phone calling pros (i.e. our team at Bland AI) create a layer of conversational awareness that manages the interplay between all three models. Additionally, to drive latency down even further, we self-host our own AI models. In-housing infrastructure enables full ownership of every output, providing greater speed and quality control.

We also create a base prompt that controls our AI’s actions. We crafted our base prompt after building hundreds of custom implementations for different customers, learning exactly which wording produces the best conversational outputs. Within our API, we expose a “task” parameter, enabling customers to insert their instructions to tell the AI how to behave during phone calls.

Developers love building on Bland because we solve all the hard problems for them. Users send us a task and a phone number to call. We manage the entire phone call and return a transcript. Building an enterprise-grade phone calling service, that fully replaces human phone callers, requires a cutting-edge set of advancements both at the model and application layer. That’s what we’re focused on at Bland AI.

Use cases for AI phone calls

Most organizations make routine phone calls. E.g. calls to vendors, customers, or employees. All those calls should be automated (using Bland).

In healthcare, low-hanging fruit includes appointment reminders, referrals, and basic data collection from insurers and pharmacies. In financial services, AI phone calls can automate account services, fraud detection, and debt consolidation and collection. In logistics, AI phone calls can automate brokerage operations. From tracking shipments to providing real-time analytics about drivers and their vehicles.

The simpler the phone call, the easier for AI to automate. Read more about other AI phone call use cases here.

That said, the fundamental difference between an AI phone call and a robocall is that the AI thinks and responds on the fly. Meaning your AI caller not only can automate routine phone operations, but it can provide high-touch, concierge service to leads and customers.

Our friends at ServiceNow recently provided a fantastic example. They did a webinar showing how to send AI phone call alerts to customers during outages. They provide the AI agent information about the outage, and let the customer ask questions to understand the issue and timeline to resolution. Especially for VIP customers, receiving a personalized phone call builds trust and drives better experiences.

Finally: did you know AI phone calls can give you an edge? Right now, there’s an arbitrage opportunity to send phone calls and build extremely valuable datasets that no one else has.

You can use AI phone calls to contact small businesses, pharmacies, schools, and government institutions. Collect information that’s unavailable online to create and augment existing datasets. It’s so powerful for everything from forecasting demand to updating financial models, to enabling private equity and real estate deal sourcing.

Implementing AI phone calls

We’ve worked with hundreds of organizations to integrate AI phone calls into their companies. We have three lessons to share.

Lesson #1: Start with a clear goal.

“Automating customer service” and “doing sales” are too broad. Scope in further. Make it simpler.

This is especially important because the AI will require a new prompt for each task. If you start with a huge, general task, you’ll have to build a mega prompt that will (probably) fail. Whereas, if you have a clear, scoped task, you can build and iterate on a simple action.

Here are some great starter tasks:

1. Remind patients to show up for their appointments
2. Qualify leads by asking these three questions…
3. Call each small business and get the owner’s contact info…

Lesson #2: Pick the right tool.

At Bland, we offer two solutions. The first is direct API access. The second is a custom implementation. Both are great options, depending on your company’s size and needs.

Developers love direct API access. They can immediately spin up phone calls and integrate them into their applications. Plus our pricing model is so simple: we charge $0.12/minute, prorated to the exact second (no shenanigans).

If you’re a developer, you should check out our API right now. You’ll get $2 free testing credits. Plus you can talk to other developers and get support via our Discord.

Many enterprises, however, want custom solutions. They want support crafting prompts, customizing voices and languages, and testing the AI to reliably achieve their chosen use case. They want to automate their customer support/engagement, lead qualification, and data collection. They need help to do it.

That’s when it makes sense to work with our solutions engineers. Together, you’ll define the task, timeline, and metric for success. Then, you’ll work hand-in-hand to build and scale that solution.

If your company requires a custom AI phone calling solution, submit an enterprise inquiry here.

Lesson #3: It's all about the prompting.

We are developers. We absolutely HATE that prompting - a fuzzy art - matters. But it does. Bad prompting creates bad phone calls. Great prompting creates phenomenal phone calls. You might want to skimp out on your prompt. You shouldn’t.

Within your prompt, include a clear overarching goal. Then include a sequence of steps to follow in the conversation. Then provide dialogue examples. Then provide any additional context the AI should have.

We wrote a whole guide, specifically for prompting. It includes an excellent prompt that you can copy/paste. Here it is.

Conclusion

Congratulations! You’re now an AI phone-calling expert. You know exactly what an AI phone call is, how it works, and how you can integrate AI phone calls into your own business.

Now, pick a call to action. Sign up for our developer portal. Or request time with our team and discuss a custom solution.

Thanks for reading - and until next time!