What a Chatbot Can't Tell You About Your Store
Ask ChatGPT or Claude why your Shopify store isn't converting and you'll get a good answer. Check your page speed, tighten your product titles, review your mobile checkout, look at your structured data. None of it is wrong. Most of it is what we'd say too.
So it's a fair question, and one we hear more often now: if the advice is free and correct, what is an audit for?
We build with these tools every day, so this isn't a piece about AI being overrated. It's about what a general model can see when it answers you, what it can't, and where the real work on a store has always been.
It Answers Without Looking
A chatbot answering "why is my conversion rate low" is working from patterns across thousands of stores. It hasn't seen yours. It doesn't have your GA4 or Shopify analytics, so it can't tell you that mobile converts at a third of desktop while mobile is most of your sessions. It can't open your theme repository and see that the sticky add-to-cart bar was commented out in a 2023 hotfix and never restored. It can't list your installed apps and notice two of them inject a review widget, or that the chat widget loads a few hundred kilobytes of JavaScript before the first product image.
It doesn't have your GA4 or Shopify analytics, so it can't tell you that mobile converts at a third of desktop.
Give it a URL and it can fetch a page, but a fetch is not a session. It doesn't scroll, tap a variant, watch the layout shift when the promo bar loads, or hit the cart on a throttled connection. Everything it tells you is plausible for a store like yours. Whether it's true of your store is the part it can't check.
That's not a criticism. It's the same limit a consultant has before they've logged in. The difference is the consultant knows to ask for the login.
Three Findings, One Problem
The pattern we see most often in audits isn't one big fault. It's three small ones that turn out to be the same thing.
A store has a Lighthouse mobile score in the 40s. Its product pages sit behind a 302 redirect chain from an old domain. And its product titles read like SKU labels. Ask a chatbot about each and you'll get three separate, sensible answers: compress your images, fix your redirects, rewrite your titles.
Look at all three together and a different story appears. The agency that built the site three years ago migrated from another platform in a hurry. The redirects were meant to be temporary and were never finalised. The titles came across from the old PIM export untouched. The image sizes are whatever the old platform happened to serve. The problem is an unfinished migration, and the fix is a proper reconciliation of the old catalogue against the new one, not three tickets in three different queues.
A general model answers the question you asked. It can't answer the question underneath it, because the context for that lives in your analytics, your theme history and your app stack, and it's never seen any of them. Connecting findings is where most of the value in an audit sits, and it's the step that needs someone with the data open.
Diagnosis Was Never the Hard Part
This is the part we'd say even if AI didn't exist. Most owners who've been through two or three agencies already know roughly what's wrong. They've been told the site is slow. They've been told mobile needs work. The reason it hasn't changed isn't a shortage of advice.
It's that "improve mobile page speed" turns into forty decisions on a heavily customised theme. Which of the eleven apps can go? Which section is the LCP element on the collection template, and does the theme let you lazy-load below it without breaking the quick-add? Is the layout shift from the announcement bar or the font swap? Can you change the product title structure without breaking the feed Google Merchant Center depends on?
Knowing what good looks like is the entry ticket.
Each of those has a correct answer, and none of them is in a chatbot's reply, because each depends on what's in your theme and what's wired into it. The wiring and the shipping is the job.
Where We Use It
We use these tools a lot. To draft the first pass of a title rewrite across 800 SKUs. To summarise a theme's section files before we open them. To write the structured data test cases. They make the mechanical half of the work faster, which means more of the hours go into the judgment half.
What they don't replace is someone sitting with your numbers, your code and your app list at the same time, deciding which of the twenty plausible problems is the one costing you money this quarter.
Go and Check
Do this before you book anyone, including us. Ask your preferred AI tool why your store's mobile conversion is low. Save the answer. Then open Shopify's web performance report and your theme's section list, and go through the answer line by line.
Mark each recommendation as:
— "already true of my store"
— "not true of my store"
— "can't tell without more digging"
The third column is the audit. It's usually the longest. If that list ends up longer than you'd like, that's what we're here for — get in touch and we'll go through it with you.
Related pieces: Five Years of Ad Spend and a Flat Revenue Line, Stop Chasing a Green Lighthouse Score, and How to Audit Your Own Shopify Store in an Afternoon.
Is your store making any of these mistakes?
We audit Shopify stores for exactly these kinds of conversion-killing patterns — the ones that have become normalised until you see them through a customer's eyes.
Book your free audit →
