Give Feedback

Different criteria for Champion / Challenger Splits

When we do A/B testing if we select Champion/Challenger as an option, we aren't allowed to designate the Champion by clickthroughs... it automatically selects the winner by opens. It would be nice if we could choose the criteria we wanted to report on.

6 votes
Sign in
(thinking…)
Sign in with: facebook google
Signed in as (Sign out)

We’ll send you updates on this idea

Lucas Weber shared this idea  ·   ·  Flag idea as inappropriate…  ·  Admin →
declined  ·  AdminBronto (Product Management, Bronto Software) responded  · 

The Champion/Challenger by definition does not select a winner for you. It has no final winning group and the testing versions send to all recipients in your target groups. Many of our customers use it for a 50/50 test or to try sending on different days. The reporting will indicate which of your versions had a higher open rate, but it is not indicating a winner, nor does this mean that version was treated any differently. If you want to have open or click rate taken into consideration when testing content you will need to use the A/B Winner split option.

3 comments

Sign in
(thinking…)
Sign in with: facebook google
Signed in as (Sign out)
Submitting...
  • Kristina Simonsen commented  ·   ·  Flag as inappropriate

    I was just chatting with Alec about this. For Champion/Challenger, the winner should be default set based on the type of tests. I've run into situations where I'm doing an A/B content test with Champion/Challenger and I set up a remail, which will send the "winning" piece of content. But the winner is based on open rate in this case, and it should be based on click rate by default. But there's no option to select winner under Champion/Challenger, so the remail will just send the piece of content with the highest open rate, which has nothing to do with which piece of content was more effective in driving click throughs.

  • Jessica Nylund commented  ·   ·  Flag as inappropriate

    It's now 2013 and this issue is still a problem. As for now we can not test anything but open rate using the Champion/Challenge function if we want to do remailing.

  • Mike Becak commented  ·   ·  Flag as inappropriate

    Select the metric you want to test on - like click rate - while "A/B Winner" is selected. Then click the button for "Champion/Challenger." The test will select the winner by the metric you chose.

Join the UX Brigade!

Bronto’s User Experience team is interested in hearing from you. Would you like to participate?

Schedule Time Now

Feedback and Knowledge Base