Identifying the Winning Variant

You can find details about variant performance on the Insights tab, including information about conversion rate, lift, count, and confidence for each variant. The count specifies the number of unique users who have seen only the variant that is associated with the count. Users who see multiple variants during an A/B test are excluded from the variant count.

In addition, you can see which data views make up your metrics.

The overall winner is the variant that performs best based on the data views that make up the primary metric. Typically, the variant with the greatest lift over the champion is recommended as the winner. The champion is recommended as the winner if the other variants have a negative lift.

The variant with the highest measured conversion rate is the baseline. In order for the variant to be declared the winner, these conditions must be met:

• The unique visitor count meets or exceeds the estimated sample size.
• The lift, when the variant is the challenger against every other variant, meets or exceeds the threshold.
• The confidence for every pair involving the variant meets or exceeds the threshold. The threshold defaults to 90%. For information about changing the default threshold in Applied Analytics, see Test in SAS Customer Intelligence 360: Administration Guide.

In addition to the overall variant winner, the analytics might find that a different variant performs better for a subset of your target audience. This subset of people is known as a discovered segment. Variants and any corresponding discovered segments are listed on the Insights tab, along with a segment profile. The segment profile contains information about the discovered segment, including these details:

• a count of how many people in the task match the segment criteria.
• a list of criteria that distinguish the discovered segment from all other people that accessed the creatives being tested. The criteria are listed in order from most distinguishing to least distinguishing.
• user-defined criteria that were made available to segment profiles when the data was imported into the data hub.

The Insights tab denotes the champion and overall variant winner for a primary metric. The champion is denoted with a . The overall variant winner is denoted with a .

The system recommends a winner, but you must click Declare Winner and select the variant that you want to mark as the winner. When a variant is declared a winner, the test task ends, although you can create a task that continues delivering the winning creative.