A/B Testing | Dan Siroker

Summary of: A/B Testing: The Most Powerful Way to Turn Clicks Into Customers
By: Dan Siroker


Embark on a journey to discover the power of A/B testing through the book ‘A/B Testing: The Most Powerful Way to Turn Clicks Into Customers’ by Dan Siroker. Delve into the world of comparative testing to optimize websites for better user engagement and higher conversion rates. Learn about the tested strategies and tactics employed by President Barack Obama’s successful 2008 presidential campaign. Throughout the summary, explore a systematic five-step process to conduct effective A/B tests, understand the benefits of data-driven company culture, and unravel the ways to involve shared methodologies and data in decision-making processes.

Obama’s Testing Strategies

President Obama’s 2008 presidential campaign set the standard for optimizing communication through A/B testing. His team tested tweets, website designs, and calls to action, ultimately increasing sign-ups by 40.6%, resulting in an additional $50 million in donations. While expensive and time-consuming at the time, testing has since become faster, cheaper, and easier, spreading to more companies. Creating a culture of testing also fosters the use of shared data and techniques for decision-making.

Purposeful Testing Process

To effectively perform A/B testing, start by defining success through quantifiable metrics and identifying bottlenecks on your site. Next, construct a hypothesis as to why these bottlenecks exist and prioritize potential solutions based on their impact. Finally, test your hypothesis and determine which version of your site most effectively turns visitors into customers. It’s important to avoid changing elements without setting clear goals or criteria. Instead, follow a purposeful and deliberate five-step process. Success can be defined through macro and micro conversions, and understanding where users are leaving your site will help you create a hypothesis and prioritize solutions. By testing your hypothesis and measuring its impact, you can optimize your site for maximum customer conversion.

Testing for Better Website Performance

Website optimization involves either refining your existing design or exploring radically different alternatives. This requires “humility and bravery” but can produce greater returns. However, it’s important to test your way to better returns rather than settling for weak solutions. A/B testing is a collaborative, interdisciplinary approach to testing specific visual elements and overall site design. Defining success in the context of A/B testing involves turning your site’s ultimate purpose into quantifiable success metrics. Often, sites become more effective when you remove elements rather than adding them. Subtraction works particularly well for optimization in the checkout funnel. Changing words can be a quick way to test many variations and fundamentally change a statement’s meaning. It’s important to test key words that call on users to act and consider how to frame your message to produce different emotional effects. Test various frames, both positive and negative, emphasizing loss or gain, and accenting passive or active responses.

A/B Testing Strategies

A/B Testing can be done using three basic methods: building your own tools, buying tools, or hiring help. Each approach has its advantages and disadvantages.

Building your own tools is best for large organizations with a technical staff or those already building analytics. Developing your own testing tools is logical if you must test proprietary systems or do extensive testing. Building your own tools doesn’t make sense for small organizations without engineering support. It is also essential to do an A/A test if you build your tools.

Buying tools through the Software-as-a-Service (SaaS) model is the most common way to acquire testing tools. In this method, no physical purchase or downloaded software is needed. Instead, you integrate the tool via a one-time copy-and-paste onto your site. Some platforms fit a wide range of needs and work for businesses of all sizes. To choose the right tool for your firm, select a platform that integrates with your existing software and fits your budget.

Lastly, hiring outside consultants to do testing is an option for those without the financial or technical resources to do it themselves. Before hiring consultants, review their reputation, compare different plans, and ensure you can afford them. Use these approaches to determine the best choice for your organization.

A/B Testing: Shifting to an Empirical Foundation

A/B testing has become an increasingly ubiquitous and critical part of doing business online. However, one practical concern is whether your organization will accept the results. Often, companies make decisions based on the “Highest-Paid Person’s Opinion” instead of data, known as the “HiPPO Syndrome.” To guide your culture towards an empirical foundation, explain testing’s two major benefits: to listen to customers more effectively and shift to a testing-based culture to remove the need for people to defend themselves and their positions.

To implement testing, begin with elements that people are willing to change and exclude issues caught up in ongoing conflict. Determine a few places you can test quickly and cheaply to include everyone in “early wins.” Share test results with team members early and update others regularly to transform them. Keep in mind that testing lets you quantify your value, which helps your position. By shifting to an empirical foundation, teams can avoid basing decisions on emotion or ideology, ultimately shifting employees’ focus and helping people get along better.

Want to read the full book summary?

Leave a Reply

Your email address will not be published. Required fields are marked *

Fill out this field
Fill out this field
Please enter a valid email address.
You need to agree with the terms to proceed