Every owner who considers an AI agent asks the same question, and it is the right one: what if it writes something stupid to my customer? Not in theory, not to a tester, but to the client who has been with you for years and now reads a reply with a wrong price in it.
There is a way to answer that question without risking a single conversation. It is called a shadow test, or test mode: the agent works on your real incoming messages, and the replies go to you instead of the customer. This article explains how such a test is run, what to look at in the report it produces, and how to fix what you do not like.
What a shadow test is
A shadow test is a period when the agent is fully connected and fully working, but muted on the outside. Real messages come in from your channels. The agent reads them, looks up your data, writes the reply it would send, and stops there. The reply is recorded for you. The customer gets nothing from the agent and keeps talking to your team exactly as before.
That makes it different from the two things it is usually confused with. A demo shows you a vendor's prepared dialogue. A sandbox lets you type test questions yourself. A shadow test uses the questions your customers actually ask, in the words they actually use, including the messy ones at one in the morning with three questions in a single line.
Why a demo proves less than you think
Scripted demos fail on the same thing in every industry: real customers do not ask the question the way the script expects. They ask for a price for something that is not on the list. They change their mind halfway. They write in a language nobody planned for. They ask for a discount because they came last year.
Those are the cases that decide whether an AI agent is useful to your business, and none of them appear in a demo. They appear in your own inbox, which is why the test has to run there.
The three steps: install, test mode, launch
At SVI every team of agents goes live in the same three steps, in this order:
- Install. The solution or the team is added to your account and your channels are connected: website chat, Telegram, WhatsApp, Instagram, email. This step is on us.
- Test mode, the shadow test. The agents answer your real messages, your customers do not see the replies, and you see every one of them plus a report. This step is on us too.
- Launch. When the replies look like the answers you would give, you switch the agents on and they start talking to customers. Payment starts only here.
The order matters more than it looks. Because payment starts at launch, nobody has to argue about whether the agent is good enough in the abstract: you read its work on your own conversations first and decide afterwards. The same three steps apply to a ready-made solution and to a team you assemble yourself in the agent builder.
What to read in the test report
A test is only as good as what you check in it. Reading every single reply is possible in the first days and pointless after that; it is better to read with a few specific questions in mind.
- The facts. Pick the replies that contain a price, a date, an address or a condition and compare them with your own data. This is where a weak setup shows itself fastest.
- The handovers. Look at where the agent decided to pass the conversation to a person. Too rarely is a risk; far too often means it is missing data it should have.
- The refusals. Find the moments when the agent said it did not know. Each one is a gap in what you gave it, and usually a cheap thing to fix.
- The tone. Read three or four replies out loud. Does it sound like your business, or like a call centre in another country?
- The requests it collected. A booking or an order should arrive complete: what, when, who, how to reach them. Half a request is work for your team, not relief from it.
- The hard ones. Complaints, group bookings, unusual demands, people asking for your boss. These are the cases where you want to see the agent step aside, not improvise.
What you will not find in a report is a grade. The test does not tell you whether to launch; it shows you what would have been said, and the decision stays with you.
What to change, and how
The part owners least expect is that fixing a reply you did not like is not a development task. You describe the change in plain words, the way you would tell a new employee, and the rule changes. No code, no ticket, no release.
- A wrong answer. Give the agent the missing data: the correct price list, the real opening hours, the condition it did not know about.
- A reply that went too far. Say where the line is: never promise a date without confirmation, never discuss other clients, never approve a discount.
- Too much or too little escalation. Name the cases a person must see, and let the agent close the rest.
- The wrong voice. Describe how you want to sound: shorter, warmer, more formal, with the greeting you always use.
Then you look at the next days of messages and see whether the change held. Several rounds of this are normal, and they are what turns a generic assistant into an agent that works your way. Rules can also be set role by role in the builder, where Mai puts a team together from 19 roles, each one arriving with ready rules you adjust rather than write from scratch.
The limits that hold whatever you configure
Some things are not settings, and that is deliberate. In every SVI solution the agents answer only from the business's own data and do not invent prices or open time slots. They never give a discount the owner has not approved, and never reveal one customer's data to another. A booking stays a request until a person confirms it. When an agent is unsure, it asks or hands the conversation to a person with a short summary.
One more limit is worth saying out loud, because it comes up in the test reports: the agent does not pretend to be a human and does not deny being an AI assistant if a customer asks. It can be perfectly natural and still honest about what it is, and most customers care far less about that than owners fear.
What you need to start a test
Less than a software project. One channel is enough to begin, for example Instagram Direct. One scenario, the one that eats most of your team's time. And 30 to 50 real conversations to check the replies against, which almost every business already has in its chat history.
It also helps to write down your own baseline for a week before launch, changing nothing: how fast you reply now, how many messages arrive after hours, how many conversations end with nobody answering. Without those numbers, the comparison after launch turns into an argument about impressions. The AI receptionist page shows what such a starting set looks like for a spa, a salon, a hotel or a clinic.
Talk to an agent that has already been through this
If you would rather see the result than read about the method, Andaman Lotus Spa is a fictional demo company with a real working agent behind it. Ask for a price, book a treatment for Saturday evening, change your mind, switch languages mid conversation, try to push it into promising something it should not. That is exactly the kind of poking a shadow test does for you, at scale, on your own customers.
How to start
Open the agent builder to see which roles your team would need and set their rules in your own words, or begin with a conversation with Mai: describe your business, and she will set up the solution and start the test mode with you. Install and test are on us, so the first thing you will be asked to judge is the agent's work on your own messages.