How to aggregate and filter results

Introduction

Emotion and attention analysis is an aggregate analysis – we collectively analyze a group of participants to look for patterns and trends in behaviour and reaction. You can select the group in Stim Engine prior to invoking the analysis. This enables you to analyze different segments of the population and compare the results.

Where to find results

Results are presented on the “Results” page, accessible from the left side menu. Once the page loads, select your Project and a Stim. Results are analyzed individually for each stim, even if multiple stims were viewed by the same participant.

Filters

Results are shown as a table with one row per participant session in which the selected stim was viewed. All sessions are shown. You will probably not want to include all in the results, even without considering demographic segmentation.

Each session has a “selected” status and a “visible” status. Visible sessions are shown in the table. Changing the filters will bring back invisible sessions. The purpose of hiding sessions is to enable the “Select all” tickbox to apply to the right subset of the sessions.

The selected status is used to determine which sessions are included in the visualizations and analyses. Note that sessions which are current invisible may still be selected. If you want to unselect all sessions, remove all the filters and untick the select-all checkbox.

We provide several filters that hide sessions from the table.

Filter

Notes

Session ID

Exact match, OR logic. This is compared to the numeric ID column in the results.

Tag filters

Must be exact match, AND logic. For example, if you enter YOUNG and MALE, then only results which are tagged both YOUNG and MALE are shown.

Name filters

Exact match, list. This value is compared to the Third Party ID which is stored in the Name column in the results.

Min/Max Tracking Accuracy

Accuracy is a root-mean-error measure. It is expressed as a percentage, where 100% is perfect and 70% is the recommended lower threshold. Accuracy is shown in the Quality column.

Tagging

Individual sessions (aka views) may be tagged. Tagging is the recommended way to build subsets of data you will to analyze in aggregate. For example, you might apply tags “HA-M” to all high-accuracy Male participants and “HA-F” to all high accuracy female participants. These two tags then provide a way to apply analyses to these groups.

Tags can be edited for individual sessions by clicking the button in the Tags column in the results table. An editor will appear.

Tags can also be edited in bulk. To add or remove a tag from a group of sessions, select them (either individually or with the select-all checkbox). The “Add/Remove” buttons will be enabled under the Tags column. Click these to enable the tag editor for the selected sessions.

Tags can also be used as a filter, as described above, to enable more efficient use of the select-all checkbox while segmenting the results.

Participant information

The results table contains several pieces of information about participants.

Column

Notes

Name (Third-party ID)

This data is used for demographic segmentation. This ID is passed from the embedding page during the session (i.e. you can customize what is recorded). Use this with name filter to segment based on demographics captured elsewhere, such as in survey questions.

Location

This is derived from the participant IP address. It consists of County / State or City, and Country.

Date/time

Server time is displayed to reduce confusion (everyone sees the same dates and times). This is the time the session was recorded.

Quality

ET accuracy as root mean square error as a fraction of screen height.

View

If views contain replayable data (including mouse recordings and HTML page reconstruction) a ‘Play’ indicator is shown. If the view contains eye-tracking data, a gunsight icon is also displayed.

Browser & screen

This information about the user’s computer was captured from their web browser.

Segmentation

The purpose of segmentation is to build subsets of sessions to be included in results for analysis. Use filters to build subsets with common tags and then use these tags to select the sessions for analysis. This ensures that it’s easy to reproduce any analysis or visual – just select the same tag[s] again. Note that an AND logic is applied to tag filters – for example, you can have a tag YOUNG applied to all participants aged less than 30, and a tag MALE applied to all male participants (regardless of age). If you apply the tag filter YOUNG MALE you will only see participants who are both young and male.

For an accurate measure of visual attention, and for heatmaps or AOI statistics, we recommend excluding views with less than 70% minimum accuracy. If the sample size is small, you can replay the individual gaze tracks to decide if additional sessions can be included – watch the red “gaze” track during the individual replay to check that tracking is good. Sometimes the track is OK, but the accuracy test was not completed properly. Note that it is very unlikely for participants to score highly in the accuracy test but have low accuracy in the stim recording; the only time this has been observed was due to participants sitting in completely unlit environments where the screen provided the only illumination of the face. In these conditions, tracking is lost when the screen becomes darker due to changes in stim content.

The statistics computed in emotion analysis are less affected by poor quality recordings, although including untrackable results will cause a reduction in the number of significant emotional reactions for a given population size.

Outputs

Currently we provide 3 types of output:

  • Area Of Interest (AOI) statistics (CSV data about which features in the stimulus get the most attention).
  • Heatmaps (rendered videos or images showing graphically where attention was focused. In future, joint emotion-gaze analyses will also be provided as a type of heatmap.
  • Emotion Aggregate Analysis (downloaded as a ZIP file containing CSV and PNG files).

These outputs are explained in separate documentation.

Datasets and Dashboard

All visualization outputs produced at the same time from a set of filter criteria are collectively called a Dataset. Datasets can be viewed in the “Analysis” section of Stim Admin.