News
Shopify Now Has Native A/B Testing
14/4/2026
For years, making meaningful changes to a Shopify storefront required a leap of faith. Redesign your homepage, reorder your product page, swap out a hero image, and then hope for the best. The numbers either moved or they didn't, and rarely did you know with any confidence why. That's the problem A/B testing was built to solve. And now, with the launch of Rollouts as part of Shopify's Winter '26 Edition, it's baked directly into the platform with no third-party app required. Here's everything you need to know.
What is A/B Testing?
A/B testing, sometimes called split testing, is the practice of showing two different versions of a webpage to separate groups of visitors at the same time and measuring which one performs better.
Version A might be your current homepage hero image. Version B might be a lifestyle shot of your best-selling product in use. Both go live simultaneously. Real visitors are split between them. After enough traffic has passed through, the data tells you which version drove more clicks, more add-to-carts, or more completed purchases.
It sounds simple because the principle is simple. The value lies in removing opinion from the equation entirely. Rather than debating internally whether a CTA should say "Shop Now" or "Explore the Collection", you test it, and you let your customers decide.
A/B testing applies across almost every touchpoint of a storefront: homepage layouts, collection page grids, product page structures, navigation menus, button colours, imagery, copy, promotional banners, and more. Any element that influences whether a visitor converts is a candidate for testing.
Why A/B Testing Matters for Ecommerce
For ambitious Shopify brands, the case for structured testing comes down to a straightforward idea: you can't optimise what you don't measure.
Better decisions, backed by data
Gut instinct has a place in brand building, but not when it comes to conversion rate decisions. A/B testing replaces assumption with evidence, giving your team a reliable basis for every storefront change you make.
Compounding improvements over time
A single A/B test might lift your conversion rate by a fraction of a percent. But ten well-run tests, stacked over six months, can drive meaningful, compounding growth in revenue without spending an additional penny on traffic.
Reduced risk when making changes
Launching a redesign across your entire store at once is a gamble. If it doesn't land, you've exposed every visitor to a worse experience before you've had a chance to catch the problem. Testing allows you to validate changes with a portion of your traffic before committing them fully.
Understanding your audience properly
Your analytics can tell you what your customers do. A/B testing starts to tell you why. Over time, a structured testing programme builds a picture of what resonates with your specific audience, insight that no industry benchmark can replicate.
Maximising your existing traffic
Most brands focus growth conversations on acquisition: more ads, more SEO, more reach. But improving the conversion rate of traffic you're already paying for is often the higher-leverage move, and A/B testing is the primary tool for doing it.
Introducing Shopify Rollouts
Until early 2026, running a meaningful A/B test on Shopify meant integrating a third-party platform: Shoplift, Intelligems, Convert, Optimizely. All capable tools, but all carrying a cost, both financial and technical. Monthly fees ranging from £50 to £500+, external JavaScript injected into your storefront adding page weight, and ongoing maintenance whenever a Shopify update caused a conflict.
That changed with Rollouts.
Announced as part of the Winter '26 Edition and made available from March 2026, Rollouts brings experimentation and scheduling natively into the Shopify admin. No app install. No additional monthly cost. No external scripts slowing down your pages.
What Rollouts Does
At its core, Rollouts lets you create a duplicate of your live theme, make changes to it, and control exactly what percentage of your visitors see those changes, while Shopify tracks performance for both versions simultaneously.
The traffic split happens server-side, before a page renders. This is a meaningful technical distinction: most third-party tools split traffic after the page loads, which introduces a flicker between versions and adds JavaScript overhead. Rollouts avoids that entirely.
From a practical standpoint, this is what Rollouts puts at your fingertips:
Scheduled changes
Save theme updates and set them to go live at a specific date and time. Running a flash sale that starts at midnight? No more staying up to manually publish changes. Set it once and let Shopify handle it.
Automatic reversion
Set an end date on any rollout and your store reverts automatically when it's done. Promotional banners, seasonal layouts, limited-time messaging: all cleaned up without a developer touching anything.
Native A/B testing
Run a live split test between your current theme and a variant, review the impact on conversions in Shopify Analytics, and select the winning version. Revenue per visitor is tracked natively as the primary metric, alongside conversion rate and session data.
Market-specific targeting
For merchants on Advanced and Plus plans, Rollouts can be targeted to a specific market. Run a regional promotion for North American buyers, test a localised homepage for European audiences, or trial different seasonal campaigns in different markets simultaneously.
You access Rollouts directly from your Shopify admin under Markets > Rollouts, or via the Rollouts button that appears next to your published theme in Online Store > Themes.
Who Can Use It
Rollouts is available to all Shopify merchants on Basic plans and above, making it accessible regardless of where you are in your growth journey. Market-specific targeting, the ability to run different rollouts for different regions, is reserved for Advanced and Plus plans.
A Few Things to Bear in Mind
Rollouts is a genuinely significant addition to the platform, and it's honest to frame it as the strong foundation it is rather than a complete experimentation suite.
Currently, you can't test global theme settings like typography or colour palettes, as changes need to be made within the theme editor's sections. Checkout testing remains a Shopify Plus-only territory. There's no audience segmentation, meaning you can't show version B only to returning customers, for example. The reporting, while useful, doesn't yet offer the depth of a dedicated CRO platform: no confidence intervals or custom conversion goals. Shopify has been transparent about plans to expand Rollouts to more surfaces, including products and discounts, in the coming months.
For merchants who have never run any formal testing, Rollouts removes the barrier to entry entirely. For brands running a mature CRO programme, it's a clean complement to existing tools, particularly useful for de-risking larger theme changes before pushing them to 100% of traffic.
Other A/B Testing Tools Worth Knowing
Rollouts is a compelling starting point, but there are scenarios where a dedicated testing platform remains the right choice. Here's a brief overview of the leading options:
SimGym - A Shopify-built app, currently in research preview, that lets you simulate storefront changes using AI shoppers before any real traffic is involved. Rather than running a live test straight away, you can see how AI personas respond to a new layout or campaign, including impact on add-to-cart rates, cart value, and navigation patterns. A useful sense-check to run before committing to a full Rollout.
Shoplift - Built specifically for Shopify, Shoplift offers more granular testing capabilities than Rollouts, including Bayesian statistical analysis, device-level segmentation, and AI-powered variant generation. Well-suited to brands running a comprehensive, ongoing testing programme who want depth beyond theme-level changes.
Intelligems - The go-to tool for price and profit testing. If you want to understand how different price points affect conversion rate and average order value, Intelligems does things Rollouts simply can't. A strong choice for brands where pricing strategy is a live variable.
Convert Experiences - An enterprise-grade platform offering advanced audience segmentation, custom goal tracking, and detailed revenue-per-visitor reporting. Best suited to larger operations with dedicated CRO resource and the need for granular targeting.
Optimizely - A full experimentation platform that goes well beyond ecommerce, including feature flagging and personalisation at scale. Typically used by larger organisations with significant developer resources and the requirement to test across multiple surfaces simultaneously.
The right tool depends on what you're testing, how much traffic you have, and how mature your optimisation programme is. Rollouts is the natural starting point for most Shopify merchants. Specialist platforms earn their place when your needs outgrow it.
The Takeaway
Shopify Rollouts is one of the most practically useful updates to come out of the Winter '26 Edition. It doesn't ask anything more of your team: no new subscriptions, no added complexity, no developer overhead. It gives you the ability to make every significant storefront change as a considered, data-backed decision rather than a live gamble.
For brands that have been putting off building a testing programme because of the cost or complexity involved, there's no longer a reason to wait.
If you'd like to talk through how A/B testing and Rollouts could fit into your Shopify strategy, we'd love to hear from you. Get in touch with the B2 team.
