Design an AI visibility audit you can repeat

If you’ve ever searched for your business on an AI tool and gotten a strange answer, you already know why an AI visibility audit matters. The problem is not just whether your name appears. It’s whether the system describes your business correctly, cites the right sources, and keeps that answer stable over time.

For a local business owner, that means separating two questions: “Can people find us in AI answers?” and “Did that answer lead to more calls or sales?” Those are related, but they are not the same. A good audit measures visibility first, then leaves room for later revenue tracking without mixing the two together.

Start with a clear entity list

Before you test anything, write down the exact entities you want to audit. This sounds obvious, but it is where many audits get messy.

List:

• Your business name as it appears on your site and listings
• Common variations or abbreviations
• The physical location or locations
• Service categories you want associated with the business
• Key people or departments only if they matter to public search behavior
• Brand names, if they differ from the legal business name

For a hypothetical example, imagine “Northside Glass & Door,” a local repair company with one showroom and two service areas. The audit should not just test “glass repair.” It should test the exact business name, the city names customers use, and the services that matter most, such as shower doors, window repair, and emergency board-up work.

This entity list becomes your baseline. If the business changes a name, adds a location, or shifts services, you can update the audit instead of rebuilding it.

Build prompt sets that reflect real customer questions

Next, create prompt sets that mirror how people actually ask AI tools for help. Don’t rely on a single broad prompt like “Tell me about this business.” That tells you almost nothing useful.

Create groups of prompts such as:

• Brand prompts: “What is Northside Glass & Door known for?”
• Local prompts: “Who repairs shower doors in [city]?”
• Comparison prompts: “Which local companies handle emergency glass repair?”
• Trust prompts: “What should I check before hiring a glass repair company?”
• Problem prompts: “My store window is broken tonight. Who can help?”

Keep the wording consistent so you can compare results over time. If you change the prompt every month, you won’t know whether the answer changed because the model changed or because your question did.

It also helps to note the prompt context: location, device type if relevant, and whether you used a logged-in or logged-out state when testing. Those details can affect results.

Score the answers with factual checks, not feelings

When you review an AI answer, score specific facts one by one. A simple spreadsheet is enough.

Track:

• Correct business name
• Correct location or service area
• Correct phone number or website
• Correct service descriptions
• Mention of competitors, if relevant
• Whether sources were cited
• Whether the cited source actually supports the claim

Use a pass/fail or correct/incorrect format. If the answer is partly right, note that separately. For example, an AI tool might correctly list the business name but mention the wrong neighborhood or describe a service the company does not offer.

That matters because “mostly right” can still cause lost trust. A customer who sees a wrong service area may move on without calling.

Track sources and changes over time

An audit should do more than collect screenshots. It should create a change log.

For each test, record:

• Date tested
• Prompt used
• AI tool or source surface tested
• Answer summary
• Facts that were correct or incorrect
• Sources cited, if any
• Notes on whether the answer changed from the last audit

If the answer changed, do not assume your marketing caused it. It might be because the model retrieved a different page, your website changed, a listing became inconsistent, or the prompt was interpreted differently.

That is why the log should focus on observations, not conclusions. Visibility is measurable. Revenue attribution usually needs a separate setup, such as call tracking, form tracking, and comparison with offline sales activity.

A repeatable monthly workflow

The easiest way to keep this useful is to put it on a calendar.

1. Test the same prompt set each month.
2. Use the same entity list.
3. Capture screenshots or exports.
4. Score facts the same way every time.
5. Note new sources, missing citations, or contradictory claims.
6. Compare month-over-month changes.

If the business has multiple locations, audit them separately. A single report for all locations can hide problems. One branch may be visible while another is missing or misdescribed.

If you need help turning this into a working process, a structured plan from /fractional-cmo or a direct conversation through /contact can be useful.

Tradeoffs to expect

A repeatable audit is valuable, but it has limits.

It can tell you what AI systems say today. It cannot guarantee what they will say tomorrow. It can show which sources appear to influence answers. It cannot prove causation in a simple way. And it can reveal errors without immediately telling you how to fix them.

That is still worthwhile. Once you know what is being stated incorrectly, you can look for the source of the mismatch: website copy, directory listings, service pages, schema, or inconsistent business profiles.

The biggest pitfall is trying to measure everything at once. Start with a small set of important prompts and a short list of must-have facts. A narrow audit done consistently is usually more useful than a giant one that gets abandoned after one cycle.

Action checklist

• Write down the exact business entities to test
• Build 10 to 20 consistent prompts that match real customer questions
• Decide which facts must be correct every time
• Create a spreadsheet or log for answers, sources, and changes
• Separate visibility observations from revenue tracking
• Review the same prompt set on a regular schedule
• Update the audit when the business changes

A good AI visibility audit is less about chasing clever tactics and more about keeping a clean record. If you can repeat it, you can learn from it. And if you can learn from it, you can make better decisions about what to fix next.