Different criteria for Champion / Challenger Splits
When we do A/B testing if we select Champion/Challenger as an option, we aren't allowed to designate the Champion by clickthroughs... it automatically selects the winner by opens. It would be nice if we could choose the criteria we wanted to report on.
The Champion/Challenger by definition does not select a winner for you. It has no final winning group and the testing versions send to all recipients in your target groups. Many of our customers use it for a 50/50 test or to try sending on different days. The reporting will indicate which of your versions had a higher open rate, but it is not indicating a winner, nor does this mean that version was treated any differently. If you want to have open or click rate taken into consideration when testing content you will need to use the A/B Winner split option.
Kristina Simonsen commented
I was just chatting with Alec about this. For Champion/Challenger, the winner should be default set based on the type of tests. I've run into situations where I'm doing an A/B content test with Champion/Challenger and I set up a remail, which will send the "winning" piece of content. But the winner is based on open rate in this case, and it should be based on click rate by default. But there's no option to select winner under Champion/Challenger, so the remail will just send the piece of content with the highest open rate, which has nothing to do with which piece of content was more effective in driving click throughs.
Jessica Nylund commented
It's now 2013 and this issue is still a problem. As for now we can not test anything but open rate using the Champion/Challenge function if we want to do remailing.
Mike Becak commented
Select the metric you want to test on - like click rate - while "A/B Winner" is selected. Then click the button for "Champion/Challenger." The test will select the winner by the metric you chose.