We are big believers in testing. We test our marketing strategies for a very good reason: so that we can get better at helping our clients succeed. Therefore, when our client, IndianaTourism, had an aggressive goal of increasing its email subscriber list we decided to try more than one method at once so we could test the results and pursue only the methods that worked.
We developed a strategy for the Indiana Tourism desktop website and mobile website involving an intercept pop-up window encouraging visitors to sign up for Indiana Tourism email newsletters. We split traffic evenly between seeing the pop-up on page 1, 3 or 5. The desktop site was already making use of a variety of sign up widgets and on-site ads that drove to a primary email sign up page, but we hoped that adding an intercept pop up would generate even more sign ups and not just cannibalize the existing tactics.
WhichTestWon.com selected this test case as a 2014 Gold Award Winner for Email Opt-ins! (As of August 2014, this winning case study is only available on the WhichTestWon.com website to premium members.)
Gallery Prev Next
The Test Case
Each time a visitor came to the VisitIndiana.com website, he/she was randomly assigned a page depth of 1, 3, or 5 to see the email sign up window. We tracked pop up impressions and conversion rates over a 4-month period. We also monitored conversions for the other existing sign up methods and site bounce rates.
In addition, once initial results came in and the success of the tactic became obvious, we added the pop up and page depth test to Indiana Tourism’s mobile site and tracked sign ups and conversion rates there as well.
Over the four-month test period, the addition of the desktop and mobile pop up intercepts resulted in +481% more than usual (compared to the pre-test period).
The Nitty Gritty
On the desktop site:
- Displaying the popup after 1 page resulted in the highest number of signups, but it did so at the lowest conversion rate (3.23%).
- Displaying the popup after 5 pages generated the least signups, but with the strongest conversion rate (4.64%).
On the mobile site:
- Similar to the desktop site, displaying the popup after 1 page generated the most signups (+94% more than 3 pages and +313% more than 5 pages), but the strongest conversion rate was achieved after 3 pages (4.53%).
- The lowest conversion rate was after 5 pages (3.07%).
Interestingly, when viewed as trend lines, the performance of all three tests (after 1 page, 3 pages, 5 pages) steadily improved during the test, both in terms of more sign ups and better conversion rates.
Since initially running this four-month test in late 2012, the pop-up has generated tens of thousands additional signups. The email sign-up levels and conversion rates of all three page depths have remained consistent all this time and have not dropped off.
Our Responsibilities: Analytics, Lead Generation
Client: Indiana Tourism
Awards: WhichTestWon.com 2014 Gold Award for Email Opt-ins