15 Ideas for A/B Testing in 2012

The three most common split tests in conversion optimization are headlines, buttons, and forms. These are tried and true simple tests which you can conduct. But if you're ready to take advantage of current design trends, new technologies, and more advanced coding language, read on to discover how to adapt your website and landing pages to the way in which modern users consume information.

This isn't an article to convince you why you should be doing A/B testing.  I'm going to assume that you've researched the topic, found some wonderful articles convincing to you embark on the magical journey that is the topic, and now you're just trying to figure out the first step.  If you do need some convincing or explanation on the subject, I recommend the following articles:

Multivariate Testing in Action: Five Simple Steps to Increase Conversion Rates (Smashing Magazine).  While focusing on multivariate testing, this article is actually very informative about testing in general.
Comprehensive Review of Usability and User Experience Testing Tools (Smashing Magazine).  Need help finding a tool to conduct and track the metrics of your tests?  This covers many of the online tools (as of October 2011).  Be aware that Google Website Optimizer was integrated into Google Analytics as Content Experiments earlier this year.

A/B Testing, Usability Engineering, Radical Innovation: What Pays Best? (Jakob Nielsen). Nielsen takes a very business-oriented look at the costs and benefits of A/B testing.  Excellent data for when you need to convince a CEO why they should invest money in A/B and multivariate testing.


Chik-Fil-A's divisive stance on gay marriage highlights how public reactions to major companies weighing in on the issue of gay marriage has become the center spotlight of 2012. Earlier this year when the pro-family group One Million Moms tried to launch a boycott against JC Penny for using Ellen DeGeneres as their new spokesperson, the pro-gay rights community rallied with an outcry of support and brand advocation. Joining the retail company in supporting gay rights are Amazon.com's Jeff Bezos (he donated $2.5 million to a Washington campaign to keep same-sex unions legal in the state), Microsoft's Bill Gates, Starbucks, Nike, General Mills, Nabisco, Google, Kraft Foods, Ben & Jerry's, and Target, each showing support with either donations, publicly voicing support, or using advertising directly target at same-sex couples.

The risks of going public on controversial issues can be risky, especially with consumers' ability to virtually organize to make their dissent heard.  However, Starbucks' CEO Howard Schulz commented on his own company's decision to publicly show support of gay marriage in March:

"From time to time we are going to make a decision that we think is consistent with the heritage and the tradition of the company that perhaps may be inconsistent with one group's view of the world."

He explained the coffee giant's pro-gay positions were about making its employees proud — and aligning with its corporate values.  Worried about the impact on profit?  Schultz continued, citing citing revenue growth in the face of efforts led by pro-family organizations, 

"Since we made that decision there has not been any dilution whatsoever in our business, and as you can see, shareholder value has increased significantly."

If you feel comfortable aligning your company with a side on the issue, showing support of the gay community can take many forms.  Try utilizing same-sex couples in your advertising photography or posting support for gay marriage or gay pride on your social networks.

Ideas for AB Testing | Target the Gay Community


Social sharing on sites like Facebook, Twitter, Google+, and Pinterest can drive more traffic to your site.  But at times, these social sharing icons can add clutter to your page if you're just haphazardly adding them.  Test which networks to include.  Test the size of the social icons.  Try different placements of social shares: at the top of the page above the fold, just above the comments / reviews sections, in your header, as fixed position / sticky bar that scrolls with the content, or even not at all.  Do you really need that "Like" button on your contact page?


Your site needs to work on a mobile device.  It's no longer a "should" statement; users are increasingly using mobile devices to consume information online.  The extent of your optimization can be decided through split testing.  Test which information you show to or withhold from mobile users and how it affects your conversions.  Test if a responsive grid is sufficient or if your complex ecommerce site needs more advanced codine.  Test the usage of touch gestures for interacting with content, such as the ability to swipe through an image gallery.   If you're using Flash elements, try animation generated through JQuery or CSS3.


In the '90s it was commonly accepted that users simply didn't scroll to content below the "fold", the magical invisible line that separated the area seen on the screen initially versus what is revealed upon scrolling.  This old mantra stemmed from the idea that the Internet was fairly new and many people were adapting to how to use a browser.  Well, put away your Prince CD, because it's not 1999 anymore and the party isn't taking place in a cramped 800 x 600 pixel room.  People know to scroll nowadays and your page design doesn't need to cram as much information as possible within a limited screen space, like within a tabbed box.  In my own landing page testing, I've discovered that people are more likely to scroll through content than click on tabs to display it.  No wonder, as modern computer mice include scroll wheels and mobile users are keen to use gestures to scroll.  Try displaying important information buried in tabs as stacked items instead.  Or on the reverse side, try bundling less important information that takes us too much vertical space within tabs.

For ecommerce sites, try taking your paginated category pages loading it in one continuous long page.  Or, on the flip side, if your giant product list is dragging down site loading speed (be sure to test this on both desktop and mobile devices), try pagination or breaking up the long scroll with a "Load More" button at the bottom of the loaded content.


A con of long-scrolling pages is the fact that users may find themselves stranded half-way through and desire a different path. The solution is the use of fixed navigation, or a menu that remains visible beyond the fold.  Test out the look of this fixed menu: exactly the same as your page's header versus a simpler version versus semi-transparent versus or a simple "back to top" icon.


Custom fonts are the weapon of choice for many web designers and nowadays are easier to use and quicker to load than in previous years.  Try replacing your standard Arial font used in your header with a stylish font.  If you're a content-heavy site, test out various font-sizes for your body.  I see many dated sites using 12px or smaller for their numerous paragraphs while more successful sites use at least 16px, a much more digestible size for both desktop and mobile devices.


While Flash is dead/dying in its web design usage, animated items on a web page aren't.  JQuery and CSS3 transitions/animations enrich user experience and while not entirely browser compatible (ahem, Internet Explorer), are more mobile-friendly and less resource-intense than Flash.  Keep in mind that while JQuery and CSS3 may not consume as much page loading time as Flash does, it does consume just as much time on the user's end.  No one wants to sit through a paragraph of text being typed out letter by letter.  Test the timing of your animations.  Test your animated image sliders: auto-start versus manual, content loading all at once or in stages, and the navigation of your slider (thumbnails, dot icons, or tabs).


Instead of creating one giant single list, try richer drop down menus with multiple columns or even images.   When creating mobile optimized site, hover isn't an option, so experiment with more mobile-friendly alternative designs.


The music, movie, and television industries have been directly influenced by the availability of free versions of their media (mostly through illegal torrents).  Some companies fought the trend, but cheaper or free media is here to stay.  Take note of this trend in the industry and follow suit by offering your users freebies, hopefully ones of value.  Try offering free 30-day software trials, free white papers, free products for large orders or repeat customers, or shipping upgrades.


Modal boxes are dialog windows that overlay the content of the page and require the user to interact with it before they can return to the underlying page.  While modal boxes do not open additional browser windows, they are in essence pop-up windows and can be as equally annoying as their predecessors.  Many optimization tests show high rates of conversions with the use of modal boxes, but be wary of whether or not the implementation of such a device also increases bounce rates and lost sales.  When implementing a modal box, test how it lauches: automatically on page load, after a certain amount of time has passed, or on a click.  Test using a modal box versus directing the user to another page or displaying an on-page message.


Geographical targeting literally finds your users.  Display content to your target based on their IP address or mobile tracking.   Try localized specials, offers, or announcements.  Try autofilling form fields related to location.  If you have a store or location finder, use the geo information to display the nearest store first.

If you are targeting international users, there are many options available for testing.  Match your page language to your visitor's country of origin.  Show the prices in the local currency.  If your service or product isn't available for certain countries, try creating anticipation with a sign-up form to receive notification upon availability.  Try different imagery based on the country's season, popular attractions, ethnicity, or holidays.  For countries with written languages that read right to left, flip elements on the page that cater to this orientation.


Creating a website that is compatible and pixel-exact in every browser and legacy version can be a time-consuming and costly process.  Before you jump head-in to creating a site that works exactly the same on IE6 as it does in Chrome, use split testing to see if this is even necessary.  Try graceful degradation instead as a fall-back option before crippling your site (and poor developers) with a pixel-exact rendering across all browsers.  Use the data from your tests to determine if those troublesome IE6 and IE7 users are profitable enough for your business to commit time and money to further development.


A modern trend in web design is AJAX form validation, in which each field is checked for errors dynamically, instead of being calculated upon submission.   Test not only the method in which you are validating your forms, but how the errors are visually conveyed to the user.  Red commonly equals errors, so stick to that convention.  Error indicators can be placed in a variety of ways next to the faulty input.


Social discovery is growing to the the number one method of finding content, already influencing how SERPs are generated.  Communication online is becoming increasingly casual and digestible.  Unless you are a highly technical or conservative B2B company, your site's content needs to shine with personality and remain sweet and concise.  Try testing out a friendlier tone of voice in your product descriptions, your company overview, your headlines, or your calls to action.  Break up the academic or dense paragraphs of information into more scannable items.  Through testing, discover if the right level of casualness for your audience.


Testimonials and reviews build trust with the potential customer and can make or break a decision to buy from your company.  Your site may have a great half-dozen reviews from what look like reputable companies or professionals, but if one simple Google Search reveals a below-average review on Yelp or Google Places, those testimonials start to look kind of shady.  Use A/B testing to determine the most effective and believable way of displaying feedback from satisfied customers.  Test out the length of the displayed testimonials.  Test how many are displayed at one time.  Try including your overall rating from Yelp, Google Places, or Angie's List.  If the testimonials are adding too much clutter to your page, try reducing these trust indicators to a grid of the logos using your product or service.