Most Shopify merchants install apps on instinct. They search the App Store, read a few reviews, like the first paragraph of the description, and click Install. Six months later, half the apps they installed are dead weight — duplicate functionality, performance drag, expired free trials they never used, or worse, apps that quietly broke and they didn’t notice.
The cost of a bad app install isn’t just the monthly fee. It’s the data permissions you handed over, the theme code that got modified, the performance hit your store took, and the migration pain when you eventually have to replace it.
This is a practical framework for evaluating apps before you install them. It takes about 15 minutes per app and saves significantly more time later.
The five-minute screen
Before doing any deep evaluation, run this quick screen on any app you’re considering. If it fails any of these, move on without spending more time:
Is it actively maintained? Check the “Last updated” date on the App Store listing. Anything more than 6 months old is suspect. More than 12 months old, assume it’s abandoned unless you have specific reason to think otherwise.
Does the developer respond? Email the support address listed on the App Store page. Don’t ask anything complicated — just “Hi, I’m evaluating your app, can you confirm support hours and response times?” If you don’t get a reply within 48 hours, that’s your answer about what support will be like when you actually need it.
Is the App Store rating above 4.0? Lower than 4.0 needs a very specific reason to overcome. A 3.8 with 1,200 reviews from someone with a known reputation is different from a 3.8 with 12 reviews from a new developer.
Is there a free plan or free trial? Without one, you can’t actually test the app before committing. If a developer won’t let you try before you buy on a $10/month app, the answer should usually be no.
If the app passes these four checks, move on to deeper evaluation. If not, find another option.
Reading reviews properly
App Store ratings are the most-used and most-misused signal. The aggregate number is misleading. What matters is the pattern.
The recent-reviews test
Ignore the lifetime average. Look at the last 90 days specifically.
An app with 5.0 stars overall but a string of 1-star and 2-star reviews in the last three months is in trouble — something has changed, and you’re about to inherit whatever it is. An app with 4.5 stars overall but consistently positive recent reviews is in better shape, even though the headline number is lower.
To check this on the Shopify App Store:
- Open the app’s listing page
- Scroll to reviews
- Click the filter to show 1-star reviews first
- Sort by “Most recent”
- Read the first 10 reviews and look at the dates
If you see a cluster of 1-star reviews concentrated in the last 30-60 days, something specific happened recently. Read those reviews to find out what.
What 1-star reviews actually tell you
Different complaint patterns mean different things. Some categories of 1-star review are deal-breakers; others are noise.
Deal-breakers:
- “Stopped working with no explanation” — operational failure
- “Support hasn’t responded in [N] days” — abandonment risk
- “Charged me after I uninstalled” — billing problems
- “Modified my theme without permission” — trust violation
- “Lost all my data” — data integrity problem
- “Found out about hidden fees later” — pricing transparency issue
Worth investigating but not automatic deal-breakers:
- “Doesn’t work on my theme” — could be a real incompatibility, could be user error
- “Too complicated to set up” — depends on your technical comfort
- “Missing feature I wanted” — they may add it; or it may genuinely not be the right app
- “Slow to load” — check the magnitude; some merchants overreact
Mostly noise:
- “Customer support was rude” — single instances are often outliers; look for patterns
- “Doesn’t do exactly what I wanted” — app might be fine, expectation was wrong
- “Couldn’t figure out how to use it” — possibly user error, possibly bad UX
The tenure signal
Pay attention to how long the unhappy reviewer used the app before leaving the review. Shopify shows this on each review (“X years using the app”).
Long-tenured user leaves 1-star review: This is the strongest possible bad signal. A merchant who paid for 4 years and then writes a 1-star review is telling you something specific went wrong recently. These are the reviews that predict app abandonment.
Brand-new user leaves 1-star review: Often noise. They might not have completed setup, might be on the wrong plan, might have unrealistic expectations.
Long-tenured user leaves 5-star review: Strong positive signal. Loyalty over time is genuine.
Brand-new user leaves 5-star review: Possibly genuine, possibly the developer asking for reviews aggressively after install.
When in doubt, weight long-tenured reviewers more heavily than new ones. They have more information.
The 1-star reviews you can dismiss
Reviews that complain about things the app doesn’t claim to do, written in obvious frustration without specifics, are typically dismissible:
“This is a SCAM! Don’t install!”
“Completely useless. Wasted my money.”
“Worst app on Shopify.”
No information, all emotion. Move on.
Specific complaints with details are worth taking seriously, even if you disagree with the rating:
“After installing, my product pages started loading 3 seconds slower. Tested with PageSpeed Insights before and after. Support said this was ‘expected.’ Uninstalled after two days.”
That’s data, regardless of whether it’s a 1-star or 3-star review.
Understanding what the app actually does to your store
The App Store description tells you what the app does for you. The technical reality of what it does to your store is harder to find but more important.
What permissions does it request?
Every Shopify app asks for specific scopes (permissions) during installation. The install screen shows them all. Read it.
Reasonable permissions for most apps:
- Read product information
- Read orders (if the app needs order data to function)
- Manage themes (if the app installs theme code)
- Manage files (if the app uploads images)
Permissions that need a specific reason:
- Customer data (PII) access — only legitimate for email marketing, loyalty, or customer service apps
- Order modification — only for fulfillment, accounting, or checkout-related apps
- Payment information — extremely sensitive, only for genuine payment-related apps
- All-scope access — almost never necessary; be very suspicious
If an app asks for permissions that don’t make sense for what it does, that’s a serious red flag. A photo gallery app doesn’t need access to your customer data. A discount app doesn’t need to read your inventory levels.
What does it install in your theme?
Many Shopify apps modify your theme. The good ones use Theme App Extensions (App Blocks), which install cleanly and uninstall cleanly. The bad ones write code directly into your theme files, which can leave artifacts after uninstall.
Before installing, check:
- Does the app use the modern “Theme App Extension” pattern? (Check the app’s documentation or App Store listing.)
- If not, what theme files does it modify?
- Will uninstalling cleanly remove all changes?
If the answer to the third question isn’t a clear yes, install on a duplicate theme first.
Test on a duplicate theme
This is the single most important pre-install step. Always:
- From your admin, go to Online Store → Themes
- Click the three dots menu on your published theme → Duplicate
- Click Customize on the duplicate (not your live theme)
- Install the app and configure it against the duplicate
- Test thoroughly on the duplicate before publishing
If anything goes wrong, your live store is unaffected. If everything works, you can publish the modified duplicate as your new live theme.
This adds five minutes to your install process and prevents most theme disasters.
Performance impact
App weight is invisible until your page speed drops, and then it’s hard to attribute. Test performance before and after install:
- Run your most important pages (homepage, top product page, top collection page) through PageSpeed Insights before install
- Install the app on a duplicate theme
- Run the same pages on the duplicate theme through PageSpeed Insights
- Compare scores, particularly LCP (Largest Contentful Paint) and CLS (Cumulative Layout Shift)
A 2-3 point Core Web Vitals drop is normal for any new app. A 10+ point drop is a problem. A 30+ point drop disqualifies the app.
Common causes of bad performance impact:
- Apps that ship 100KB+ of JavaScript on every page (even pages where the app isn’t used)
- Apps that load fonts or icons from external CDNs
- Apps that block rendering while waiting for their own API to respond
- Apps that don’t lazy-load their images
Some performance impact is unavoidable. Apps that add functionality necessarily add code. But the difference between a well-built app and a poorly-built one is significant — sometimes 10x in JavaScript size for the same feature.
Pricing — the parts that aren’t on the listing
App Store pricing pages tell you the headline. They don’t tell you the full picture.
Read the pricing in detail
Specifically:
- Is the price per month or per year (annual discount or trap)?
- Are there usage-based fees on top of the base price?
- Do features get locked behind higher tiers as you grow?
- Is the free trial actually free, or does it require credit card upfront?
Some apps appear cheap and then escalate as your store grows. A $9.99/month app that doubles when you cross 500 orders is a $19.98/month app — price it honestly when comparing.
Check for hidden costs
Things to look for:
- Branding fees: “Remove our logo” being a paid feature is common.
- Email/SMS overage: Marketing apps often charge per message above a quota.
- API call costs: Apps that integrate with external services sometimes pass through API fees.
- Setup fees: Rare on the App Store but exist.
- Migration fees: If you’re moving from another app, check whether migration is free or costs extra.
Calculate the 12-month TCO
For any app you’re seriously considering, calculate the total cost over 12 months at your expected usage. Compare apps on this number, not on the headline monthly price.
A $5/month app with a $50 setup fee and $1 per 1000 emails is genuinely cheaper than a $9/month app with no extras only if you send fewer than 4,000 emails per month over the year.
Asking the developer questions before installing
Once an app passes the screens above, email the developer with specific questions. Their responses (and response time) tell you a lot.
Good questions to ask:
- “What happens to my data if I uninstall?”
- “Where are images/data stored — on your servers or in my Shopify account?”
- “Will the app continue working if your service goes offline?”
- “How do you handle theme conflicts on [specific theme name]?”
- “Can I export my configuration if I switch to another app?”
A developer who answers all five clearly and quickly is signaling confidence in their product. A developer who responds with marketing copy or asks why you want to know is signaling something different.
The post-install checklist
You installed the app. Don’t stop evaluating. Within the first 30 days:
Week 1:
- Run PageSpeed Insights on key pages, compare to your pre-install benchmark
- Test on mobile devices, not just desktop browsers
- Check the app’s behavior on at least 3 different page types
- Confirm the app’s analytics dashboard shows reasonable data
Week 2:
- Send a non-urgent question to support, time their response
- Test what happens when you toggle the app off temporarily
- Check that the app doesn’t trigger any warnings in Shopify’s admin
Week 3-4:
- Confirm billing is correctly set up at the tier you expect
- Review any analytics or dashboards the app provides
- Make sure the app handles edge cases (out-of-stock products, unusual order types, etc.) correctly
End of month:
- Decide: keep, downgrade plan, or remove
- If removing: actually remove it. Don’t leave dormant apps installed.
Red flags that should fail an app immediately
Some signals should disqualify an app without further evaluation:
The “free with branding watermark” trap on a high-visibility app. Your customer sees the app’s logo on every page. Pay the monthly fee or pick a different app — don’t put someone else’s branding on your store.
Apps that ask you to leave a 5-star review for benefits. Anywhere on the App Store this happens it’s against Shopify’s policy. Apps that do this are managed by developers willing to break rules.
Apps with reviews complaining about being held hostage. “They threatened to delete my data unless I left a positive review” or “I uninstalled but they’re still charging me.” Move on.
Apps that don’t have a clear privacy policy. Required by Shopify, but quality varies. If you can’t find or understand the privacy policy, that’s data about how the developer treats privacy.
Apps from a developer with multiple low-rated apps. Check what else the developer has shipped. A pattern of abandoned apps from one developer predicts what will happen to the one you’re considering.
A specific case study: how the Cozy collapse could have been predicted
This is uncomfortable to mention but worth analysing because it illustrates the framework.
Cozy Image Gallery, before its collapse in mid-2025, would have passed most of the screens above. Multi-year track record. Healthy aggregate rating. Established developer. Reasonable permissions.
The signals that would have caught it:
- Reduced shipping cadence in the year before collapse. The “Last updated” date had been creeping outward. Less than 6 months wasn’t a deal-breaker, but the trend was visible if you looked.
- Support response times stretching. Reviews from late 2024 onward started mentioning slower support. Individually dismissible; in aggregate, a signal.
- Increasing 1-star tenure values. Long-tenured customers leaving 1-star reviews started appearing months before the major outage. Each one was a leading indicator.
- Single point of failure architecture. Images on Cozy’s CDN, code loaded from Cozy’s servers. The architecture meant any operational problem affected every customer simultaneously.
None of these signals individually would have predicted the collapse. Together, they painted a picture that was visible to someone looking for it.
The framework in this article wouldn’t have caught Cozy at install time in 2018. By 2024, it would have flagged enough warnings to suggest evaluating alternatives proactively rather than waiting for a crisis.
When to use this framework vs just installing the app
Be honest: not every app deserves 15 minutes of evaluation.
For low-stakes apps (free utilities, simple widgets, things you can uninstall in 30 seconds if they don’t work), skip most of this. Install, try for a day, decide.
For apps that touch your customer experience, your data, or your performance — gallery apps, email marketing, reviews, popups, checkout extensions, anything customer-facing — use the full framework. The downside risk justifies the upfront time.
For apps you’re considering long-term (12+ months of expected use), use the full framework plus actually email the developer. The signal you get from that interaction predicts how the relationship will go.
What this means for choosing a gallery app
We’re a gallery app developer, so we’ll be direct: this framework happens to favor apps with the architectural choices we made. That’s not coincidence — we built A1 Image Gallery after watching merchants get burned by apps that didn’t meet these criteria.
If you apply the framework honestly to the Shopify gallery app market, you’ll end up with a short list of two or three credible options. We’re one of them; Robin PRO is another. Compare them on the criteria above, including ones where we have weaknesses (Robin PRO has more polish in their mobile gestures than we do in v1, for instance). Then pick.
The framework matters more than which specific app you end up choosing. An app that survives this evaluation is more likely to be a good long-term partner than one that doesn’t.
Questions about evaluating a specific app, even if it’s not ours? Email [email protected]. We’re happy to give a second opinion.
A1 Image Gallery is built by A1 Local, an independent Shopify development studio based in Perth, Australia.
