The 11 Best A/B Testing Tools in 2024

Last Updated on March 7, 2024 by Alex Birkett

A/B testing (i.e. running a controlled experiment, normally in a digital environment like a website or an application) is prevalent nowadays.

All of the biggest tech companies you know – Microsoft, Netflix, Booking.com, Google – run thousands of experiments per year. Now it’s weird if you don’t run A/B tests.

There are dozens of tools on the market dedicated to A/B testing, in addition to hundreds of other marketing solutions that have built-in A/B testing as an integrated feature.

This means you can run experiments on landing pages, Facebook ads, product features, sales scripts, customer service prompts, and or literally any other user behavior in a business context.

You can optimize the conversion rate on a given web page, but you can really experiment anywhere nowadays.

You do, however, need the infrastructure to do so, meaning you’re almost always going to go through the shopping and review process to find a testing tool.

While there is no “best” A/B testing tool, there are better or worse solutions depending on your specific context.

The 11 Best A/B Testing Tools

  1. Conductrics
  2. Optimizely
  3. Convert
  4. VWO
  5. SiteSpect
  6. Kameleoon
  7. AB Tasty
  8. A/B Smartly
  9. Wasabi
  10. Google Optimize
  11. Adobe Target

1. Conductrics

Conductrics is my favorite experimentation platform. I like the CEO, Matt Gershoff, and he’s been an excellent industry mentor and friend to me. He is also one of the most thoughtful leaders in the analytics and optimization space, so you can trust the direction of the platform.

He has also built a really great piece of software that, in many ways, was far ahead of its time and definitely ahead of most of the others on the list.

Power/feature-wise, it’s the best you can get. You can run A/B tests (client or server side), multivariate experiments, and bandit tests, all from a WYSIWYG visual editor (if you want). You don’t need to install a script until you set things live, as you can use the browser extension to edit and preview variants.

Additionally, the tool lets you run predictive targeting to exploit different segments of your audience. Basically their AI/machine learning will discover which combinations of visitor traits respond best to each of your content variants, automatically.

Definitely a tool for true optimizers.

2. Optimizely

Optimizely is the biggest brand names on the list, and I have to say, I’ve always loved using the platform. It made javascript, client-side A/B testing world popular. It’s what I learned on, so I’m definitely most comfortable in this interface, even to this day.

Feature-wise, they’ve really improved in the last few years, especially in their enterprise offering. They’ve now got ‘full-stack’ experimentation capabilities, valuable for those who want to run server-side experiments or run tests on IoT, TV, or mobile apps. The also offer unlimited free feature flags and rollouts.

Overall, it’s a robust and powerful enterprise solution that even features a fully fleshed out program management platform (though unfortunately the company doesn’t really maintain support for smaller companies nowadays – it’s really only at tool for large brands).

Their stats engine is based on sequential testing, which is often more efficient, but I do love a bit more flexibility in terms of statistical analysis.

Optimizely is probably in that tier of software where, like IBM, ‘you won’t get fired for buying Optimizely.’

3. Convert

If you want almost all of the features of Optimizely (with the addition of truly wonderful customer support and a free CXL Institute membership), but at a fraction of the price, Convert is the way to go.

For starters, the interface will feel familiar and intuitive to experienced experimenters. For newbies, too, it’s not a cumbersome tool to learn. It’s got a great WYSIWYG editor, and of course all of the typical HTML, CSS, & Javascript editing tools that serious optimizers use in the testing process.

Convert has all the typical functionality you’d expect from a client-side A/B testing tool: A/B tests, split page tests, multivariate, and personalization.

What I like most about Convert is the founder and the team clearly truly care about the industry and their customers. They’re much more transparent about their features and the market, and I find the lack of marketing spin quite refreshing (see: their feature comparisons).

Plus they offer a free 15 day trial, so you’ve got nothing to lose from checking it out and seeing if you like it. I’d recommend those currently only looking at Optimizely or Adobe to take a look and see if Convert.com might meet your needs as well.

4. VWO

VWO is similar to Convert in that it’s much more affordable than Optimizely (though VWO has gotten more expensive in recent years), and it has many of the fundamental features you’d want. Of course, you can run A/B tests, split path tests, multivariate tests, and personalization targeting.

You can also use their other CRO features, things like surveys, heatmaps, session recording replays, funnels (and the rest of HotJar’s feature list, basically). If you like all-in-one, VWO is the way to go (they even offer solutions for server-side and mobile testing).

They offer other less-CRO-related tools, as well, like push notifications.

Overall, their CRO-suite is top-notch and feature rich, and I love using VWO. They offer a free 30 day trial, so again, you’ve got nothing to lose from checking it out. VWO is an excellent option for optimizers at all sizes of companies.

They also recently launched a free plan that is somewhat limited, but should work well for up to 50,000 visitors.

5. SiteSpect

SiteSpect has been around the space for a while, is well-respected, and is generally thought of as the go-to server-side A/B testing software. They do, however, offer many other testing solutions, including your typical client-side website testing.

Additionally, they offer great ecommerce optimization features like behavioral targeting and recommendation engines.

Performance-wise, this seems to be one of the fastest testing tools on the market. It’s also got a reputation to this day of being one of the most powerful and robust platforms, giving in less often to randomization errors, flicker effect, and other factors that could affect external validity.

6. Kameleoon

Kameleoon is a great experimentation platform hailing out of Paris, France but serving customers everywhere. They are known for their powerful personalization capabilities, but you can get all the basics done as well (AB, MVT, etc.).

As part of their personalization offerings, they have predictive targeting (let the machine find proper segments to exploit), business logic targeting, email personalization, content personalization, and segment analysis. They also have a feature called KCS (Kameleoon Conversion Score) that weighs the relative probability that someone will convert (likely a productized version of propensity modeling).

It’s one of the more robust personalization packages, and top it off with a trustworthy A/B test delivery system, and you’ve got a great tool.

Seeing as personalization is one of the easier things for marketing leaders to talk about and imagine, and one of the harder things to actually implement in a meaningful way, this platform could help you walk the talk.

7. AB Tasty

AB Tasty is another France-based experimentation platform, this one very powerful and feature rich.

For marketing teams, they’ve got your basic client-side A/B testing and multivariate experiments, as well as personalization and social proof notifications for online retailers.

For product teams, you’ve got robust server-side experiment delivery, feature flags, and progressive rollout. They recently launched a whole new “suite” for product teams called Flagship, marking their seriousness in providing full stack experience optimization capabilities and supporting product teams and engineers.

Great full-stack experimentation and optimization platform.

8. A/B Smartly

A/B Smartly is an experimentation platform for much more sophisticated product teams. It comes from the team that built Booking.com’s experimentation platform. The CEO, Jonas Alves, rebuilt Booking’s Experiment Tool in 2008, before becoming Head of experimentation.

Lukas Vermeer, which took over from Jonas in 2015 is also an advisor to the company. A/B Smartly is based on what the team built at Booking.com, but much more generalized. Georgi Georgiev, the author of the A/B testing book “Statistical Methods in Online A/B Testing” is also advising on the development of their product, especially when it comes to statistical design and analysis of tests.

These are some of the biggest differences between A/B Smartly and other vendors:

  • The training offered by A/B Smartly is included with the subscription and is given by people with a ton of experience (10+ years) in running experimentation at scale across other organizations
  • A/B Smartly has a very knowledgable and strong team in the experimentation field that is available to help you in real-time over Slack or Video calls. All the customers talk about the amazing support.
  • A/B Smartly is not only an A/B Testing tool. It’s also a knowledge base and communication tool. Everything is thought on how to share information across the organization and how to find that information later to make better decisions in the future.
  • A/B Smartly has really powerful data deep-dive capabilities. It’s not only possible to segment the data on anything, but it’s also possible to filter any report on any specific audience after the fact. This is crucial for debugging results and avoids having to create custom reports internally when the tool doesn’t answer all the questions.
  • A/B Smartly is rolling out features like interaction detection, variance reduction, causal inference for tests that can’t be randomized, multi-stage triggering, and other advanced features that help you run better quicker tests and only exist on the platforms of the most sophisticated tech companies like Microsoft, Booking, Netflix, Meta, etc.

9. Wasabi

Wasabi is another open-source experimentation tool, this one by Intuit. It’s simple to set up and 100% API driven (instrument in whatever language you’d like).

This one also comes with a nice, flexible UI too get you started quickly, in addition to real time metrics and reporting.

Also, look at it like this: it was built for and is used by one of the more sophisticated experimentation teams in tech, Intuit. Clearly it’s powerful and flexible enough for their myriad of products and use cases, so I’d imagine you could make it work for your org.

10. Google Optimize

Google Optimize (the free version) is a free, yet very limited, experimentation tool. It lets you run simple client-side A/B tests, as well as multivariate tests and split page tests, though you’re capped at 5 tests per container. You can seem to get past that by spinning up multiple containers (you could in the past at least, haven’t used in a while).

For some reason, Google Optimize uses both opaque observational units and an opaque statistics engine (some flavor of Bayesian, I’ve gathered). It’s some weird amalgamation of users and sessions, and totally unclear to me still.

I like the tool as a cost-free solution to randomize users in experiments, but if you’re using it, I’d log events to an external data tool and do your analysis there.

The biggest benefit to Google Optimize is the native integration with other Google tools (namely Adwords and Analytics). This allows you to push variants to custom Google Analytics segments and remarket them with display ads (just one idea). The integration is really powerful.

I can’t speak to the paid version, but the free version is great for getting started in conversion optimization (conversion rate optimization) but probably shouldn’t be used past a certain level of program sophistication.

Note: Google Optimize unavailable as of Sept 2023. Here’s a list of alternatives.

11. Adobe Target

Adobe Target is the enterprise powerhouse of the experimentation world. Even today, they still power most of the largest brands and websites, and they still have some of the most robust features of any testing tool.

As with Google Optimize, Adobe Target’s strongest benefit comes when you’re using the rest of Adobe’s Marketing Cloud, particularly Adobe Analytics. Again, you can push segments to be targeted via specific variants and also analyze your tests more deeply in Adobe Analytics.

This creates powerful potential for personalization.

Candidly, I don’t really have experience using the platform, though I know a ton of people who use it. I’ve also heard great things about the platform (especially its targeting and segmentation capabilities) from smart people I respect like Andrew Anderson. However, I don’t have direct experience on it, so I’ve gotta throw that out there.

In fairness, if you’re considering Adobe Target, a listicle probably wasn’t going to sway you away from or towards the tool anyway.

Conclusion

There’s no shortage of A/B testing tools on the market, and there’s something for everyone.

If I’m working on a large website and have an abundance of resources to work with, I’m choosing a tool like Conductics (or perhaps, given the organization, Optimizely, Adobe Target, or Google Optimize 360).

If I’m looking for a more affordable, but still powerful solution, Convert, VWO, SiteSpect, Kameleoon, and AB Tasty are all absolutely fantastic tools. They all have their nuances, but I can recommend any of them.

For sophisticated product experimentation and company-wide experimentation, I like A/B Smartly.

There are also miscellaneous solutions not on this list because I have a bit less experience with them. For one, any landing page builder should have a built in A/B testing function (for example, Unbounce). Crazy Egg also apparently has a built in A/B testing platform to complement their heatmaps. Then, there are platform specific tools, such as the WordPress A/B testing solution and personalization tools from Nelio plugin.

Finally, there are enterprise tools that are still heavily used, such as Monetate and Maxymiser. I don’t have an opinion on them because I haven’t really used them, having mainly worked with small business websites (before HubSpot at least)

Tons of A/B testing tools, no excuse not to run tests!

Alex Birkett
Alex Birkett is a product growth and experimentation expert as well as co-founder of Omniscient Digital, a premium content marketing agency. He enjoys skiing, making and experiencing music, reading and writing, and language learning. He lives in Austin, Texas with his dog, Biscuit.

One Response to “The 11 Best A/B Testing Tools in 2024

  • John Bacon
    4 years ago

    Don’t forget about Sixpack by SeatGeek!

    100% free and open-source, server-side or client-side depending on your choice, and works with any language.

    I’m still frustrated by how I haven’t found a better replacement in my role. Pay tens of thousands of dollars for a poorer and less flexible client-side experience in Optimizely instead? Nuh uh, that deserves to get you fired.