What to ask an AI phone vendor before you pay them anything

Every demo you sit through will go well. The vendor picked the scenario, the caller speaks clearly, and nothing strange happens. Then you sign, and three weeks later somebody calls from a gas station forecourt with wind on the line and says the thing in the basement is leaking again, and that is when you find out what you actually bought. I have built these systems for plumbing, HVAC and electrical shops, so this is a list I would rather you used than skipped. Here are the questions, roughly in the order I would ask them, and what a bad answer sounds like.

01Ask to hear a call that went badly

A demo is a performance. Not a dishonest one, but the vendor chose the call, the caller and the moment, and nothing about it tells you how the thing behaves on an ordinary bad Tuesday.

So ask for three specific recordings. One where the caller was hard to understand or the line was rough. One from a business in your trade, because a call to a plumber at eleven at night sounds nothing like a call to an office. And one where the system gave up and handed the caller to a human.

The reaction to that request tells you more than the recordings do. A vendor who listens to their own calls has a messy one ready, because they have been through them hunting for failures. A vendor who has to go looking has not been listening. And a vendor who says recordings are not available for privacy reasons is quietly telling you that you will not be able to hear your own either.

Then do the part they cannot script. Call the demo number yourself, standing outside next to traffic, and say what a real customer says. “Yeah hi, it’s my mum’s place over on, hang on, my son has the address, it’s the water heater, it’s making a noise.” Listen to what it does with that. While you are on the line, count the rings before it picks up, because that number is the whole product. I wrote out what a good call to a new caller actually sounds like in this breakdown of the script, and it is a fair standard to hold a vendor to.

Speed is worth being honest about here. The published research measures web inquiries rather than phone calls, so treat it as the shape of the behavior rather than a number about your line. Harvard Business Review timed how 2,241 US companies answered inbound inquiries and found only 37% replied within an hour and 23% never replied at all. On a phone that whole curve is compressed into about four rings, because the caller is not judging your response time, they are deciding whether anything happened at all.

02What does it do when it does not understand?

This is the question that separates vendors, and almost nobody asks it, because every demo is built around the call where everything is understood perfectly.

Ask what share of calls end as a booked job and what share end as a message texted to you. If the answer is that nearly everything gets booked, treat that as a warning rather than a selling point. It usually means there is a phantom booking rate nobody has measured, and you will find it the morning a truck drives forty minutes to an address that does not exist.

Then ask what makes it stop and take a message instead. Good answers sound like rules you could write down. The address came through half heard. The caller went vague about the day. The person on the phone is a tenant who cannot authorize work on the building. Bad answers sound like “the AI is smart enough to work it out,” which is a sentence that has never once been true at eight on a Saturday. I went through why confirmation has to happen on the call itself in why an AI should never book a job without confirming it.

Ask to see a real message too, not a screenshot from a marketing page. It should be short enough to read at a traffic light. Name and number, the address as the caller gave it, what is happening in their words rather than tidied into a summary, and a link to the recording so you can hear the thirty seconds that matter.

03Who answers when it breaks on a Friday night?

It will break. Something will. Their servers, your carrier, some connection between the two that neither of you controls.

The question is what your customer hears during those minutes, and there is a right answer. The call should ring at your phone the way it did before any of this existed, and if nobody picks up it should land in your own voicemail. That is the floor, and the floor is your current situation rather than something worse than it. The reason it works that way is that the routing lives in your carrier account, not in the vendor’s system, which I explained properly in what happens to your phone number.

A wrong answer is a dead line, a fast busy signal, or a recording saying the number is not in service. Ask what the caller hears, not what the dashboard shows. Those are different questions and vendors sometimes answer the easier one.

Then ask who you call at seven on a Friday evening, and get an actual number. Ask when the last outage was and make them give you a date. “We have never had one” from a company running real call volume means they are either new or not watching. Everybody who runs phone infrastructure has an outage story. The ones worth buying from tell it without being asked, including what they changed afterward.

While you are there, ask how many people work at the company. It is a fair question and the answer shapes everything else. A two person operation can be excellent and can also be one holiday away from nobody answering your email.

04What am I actually being billed for?

The price on the page is a floor. What you want to know is what the meter counts.

Per minute, per call, per booked job, flat monthly, or some mix of those. Then ask the follow up that catches people out: what counts as a call? A robocall that got answered and hung up on. A wrong number. A two second hangup. Your own tech ringing the office. An existing customer asking about an invoice. Some vendors bill every one of those, some bill none of them, and the difference on a busy month is real money.

Ask whether your biggest bill lands in your busiest month. On most per minute pricing the answer is yes, which means you pay the most for coverage in the weeks you were least able to reach the phone yourself. That is not automatically wrong, it is just worth seeing coming. I put the three options side by side with real invoices in answering service, AI, or voicemail.

Ask what setup costs and who does the work. A fair answer includes your own time, because somebody has to decide what the thing asks a caller, what counts as an emergency at your shop, and which human number an emergency goes to. Nobody can do that part for you.

Then ask for a sample invoice from a real account with the name removed. A vendor who will not show you one is asking you to sign up for a bill they are not willing to describe.

05How do I leave?

Ask this before you sign, while you can still walk away without it costing anything.

Start with the contract. Month to month or a year. What the notice period is. What happens if you cancel in month three, and whether that is a fee, a remaining balance, or nothing at all. Get the answer in writing, because this is the one that gets remembered differently by the two people in the room.

Then the number, which is the part owners are most nervous about and should be the simplest. Nobody needs to take ownership of your phone number to answer calls on it. Conditional forwarding is a rule inside your own carrier account, you turn it on with a code from your own handset, and you turn it off the same way. If cancelling involves the vendor doing something to your number, ask why, because the normal setup does not require it.

Last, your data. The recordings, the transcripts, the call log, the phone numbers of people who called you. Ask whether you can export all of it, in what format, and how long it survives after you cancel. Two years of call history is a genuinely useful record of your own business, and you do not want to find out it was only ever a screen in somebody else’s dashboard.

06The four boundary questions

These are the ones that decide whether you end up embarrassed in front of a customer.

Does it ever diagnose or quote a price? The right answer is never, on both. Every trade has jobs that sound identical on the phone and differ by a thousand dollars once the panel comes off, and a number given by something that has not seen the work is an argument you will have later at your own expense.

What does it do with an emergency? On a gas smell, a medical situation, or live electrical, the behavior should be to stop, tell the person to get out and call the gas company or 911, and hand the call to a human number. Ask to hear that exact recording. If they do not have one, they have not tested it, and you do not want your customer to be the test.

Does it say it is an AI when somebody asks? Ask them to play you the call where a caller says “am I talking to a robot?” Anything other than a straight yes is a problem, and not only an ethical one. People can tell. The ones who feel handled tell their neighbours about it.

Does anything ever dial out from my number? For an inbound answering setup the answer is no, and the reason to care is legal rather than aesthetic. In February 2024 the FCC clarified that calls using AI generated or artificial voices fall under the TCPA and generally need the called party’s prior express consent before you dial them. A call the customer placed to you does not raise that question, because they did the dialing. Outbound is a different product with different rules, including a calling window of 8am to 9pm in the called party’s local time, and you should know which one you are being sold. While you are on the subject, ask how recording is disclosed, and check your own state’s rule rather than assuming the vendor has handled it, because some states require every party on the call to consent rather than just one.

Frequently asked questions

Is it reasonable to ask for references from other contractors?

Yes, and ask for one in your trade and roughly your size. A four truck plumbing shop and a fifty employee HVAC company have different problems, and a reference from the wrong one tells you very little. When you get the owner on the phone, skip the “are you happy with it” question and ask two better ones: what went wrong in the first month, and what do you still do yourself that you thought this would handle.

The vendor says their AI is trained on my business. What does that actually mean?

Usually it means they wrote down your services, your hours, your service area, what counts as an emergency, and which number to hand off to. That is worth having and it is not magic. The useful follow up is who maintains it when you add a service or change your after hours number, how long that change takes to go live, and whether you can make it yourself or have to file a request and wait.

How long should a trial run before I can tell?

Long enough to cover one full cycle of your week, including a weekend and at least one evening, because the calls that make the case are the ones arriving when nobody was going to answer anyway. Two weeks is usually enough to hear the pattern. What you are listening for is not the good calls, it is how many messages arrived that you would have missed entirely and how many of them turned into work.

What if a vendor will not give me recordings?

Walk. Not because it proves bad intent, but because a system you cannot audit is one you will stop trusting the first time a job goes sideways, and at that point you have no way to find out what was actually said. Totals on a dashboard are not evidence. A recording is.

Should I care about uptime percentages?

Only a little. A published uptime figure is a marketing number unless someone will tell you how it is measured and what they owe you when they miss it. The question that matters more is the one from section three: when their system is down, what does your caller hear? If the answer is that the call rings at your phone as it always did, the uptime number stops being the thing you are betting your business on.

The bottom line

None of these questions are clever. They are the ordinary ones you would ask a subcontractor before handing them a key to your customers, and the only reason they get skipped with software is that the demo is polished enough to feel like the audit already happened. It did not. The demo was the sales call.

These days we book appointments for roofing companies: we find homeowners looking for roof repair or replacement, answer their replies and book them into your system. If you run a roofing company, see how it works at rushdial.com/roofing.