Implicit – Results and interpretation

Implicit Testing Concepts

Please see this article for a background on the purpose and objectives of implicit testing, including what we’re trying to measure.

Finding your results

In the “Edit Project” page, click the “Results” button.

Results are presented separately for each Trial.

We present aggregate scores (most important) and individual scores (primarily for verification and deeper dives into unexpected outcomes).

You can download both aggregate and individual results as CSV files.

In general, the speed of reaction indicates the strength of an opinion about an association (faster is stronger). The answer (Y or N) tells you whether participants agreed with the association or not. For this reason, results are aggregated for Y and N answers separately.

Since we want to measure the effect of content on associations, we aggregate each Segment separately. For example, you can see results from before and after viewing some content.

Aggregate Results

For each Implicit Test, there are two variables of import.

  1. What proportion of participants said that, yes, the Brand equals the Trait (e.g., McDonald’s is healthy)?
  2. How quickly did people respond – compared to their individual response time baseline?
  3. “Samples” section records how many respondents went through the trail out of the number of “Total” respondents. “% w. Ans.” calculates this ratio.

For #2, we don’t look at absolute response times because these can differ by various factors (e.g., participant’s age, pre- versus post-testing, etc.). Instead, we look at response times relative to a baseline of other Brand-Trait pair Trials. This relative response time is calculated as a variable, standard in statistics, called Cohen’s D. It tells us if people were quicker to respond for the Trait-Brand pair Trials compared to the baseline (a positive value), if they were slower to respond for the pair Trials (a negative value), or if there was no difference (a value of zero). To be specific, Cohen’s D is interpreted as follows:

  1. Below -0.8: Much slower than the baseline
  2. Between -0.8 and -0.5: Moderately slower
  3. Between -0.5 and -0.2: Slightly slower
  4. Between -0.2 and 0.2: No difference
  5. Between 0.2 and 0.5: Slightly faster
  6. Between 0.5 and 0.8: Moderately faster
  7. Above 0.8: Much faster than the baseline.

Looking at Figure 7, this tells us that, after viewing the Mercedes ad, more people said that Mercedes is exciting, and people were quicker in their answers. Before the ad, people generally thought Mercedes wasn’t exciting but they had to think about it; after the ad, more people thought Mercedes was exciting and they were more sure about their answer.

Individual results

On an individual basis, the Implicit Test tells us:

  • Did the participant answer yes or no (e.g. is McDonald’s healthy or not)
  • How quickly did they respond in absolute terms (milliseconds)
  • How quickly did they respond in relative terms (Cohen’s D; see above)
  • What ‘band’ did their relative response time fall into, e.g., moderately faster than the baseline (see above)

“Band”

Results are assigned to a “band” for easy reference. The band is a value from -3 to +3 with 0 being no significant effect.

  • +3: Much quicker than baseline
  • +2: Moderately quicker
  • +1: Slightly quicker
  • 0: Same as baselines
  • -1: Slightly slower
  • -2: Moderately slower
  • -3: Much slower than baseline