Deciding Which Type of Test to Run

Types of Tests

The objective of a test task is to find the variant that performs best for the test target audience. In addition, SAS Customer Intelligence 360 can also discover subsegments of the audience that respond better to a variant that is not the winning variant. For more information about Segment Discovery, see About Segment Discovery.

There are different ways to test creative performance:

A/B Test (Single Spot)

• Use this task to test how different creatives perform in the same spot. In this case, each creative is a variant in the test.
• Optimizing this test makes the variant weights dynamic, which enables SAS Customer Intelligence 360 to change the distribution of variants as the A/B test progresses. As a result, the best-performing variant gets a larger portion of the distribution.

For more information about A/B testing, see About A/B Testing.

A/B Test (Spot Set)

• Use this task to test how different combinations of creatives perform. In this case, you select a spot set to test, instead of a spot.
  Note: All spots in the set must be on the same page.
• When testing a spot set, a single combination of creatives is one variant in the test.
• Because you set up the variants manually, creating an A/B test using spot sets is most useful if you do not have too many combinations to test.
• Optimizing this test makes the variant weights dynamic, which enables SAS Customer Intelligence 360 to change the distribution of variants as the A/B test progresses. As a result, the best-performing variant gets a larger portion of the distribution.

For more information about A/B testing, see About A/B Testing.

Testing a Spot or Set of Spots

Single-spot A/B test tasks are suitable for changing one variable at a time. For example, an A/B test might determine which image out of four possible images performs best in a spot. The A/B test randomly divides the target audience so that each audience subset sees one of the four images in the spot.

If you want to vary both the image and the text associated with it, you can perform several sequential A/B tests to isolate the effect of each change. In the first A/B test task, you could have the same text with different images. In the next A/B test, you might test the winning image with four different text colors.

Here are the disadvantages of sequential A/B tests:

• They require a lot of time.
• They cannot compare the effect of the interactions of the content.