Eye Tracking Accuracy
Accuracy score for sessions
Once you have some sessions through the StimEngine platform you’ll notice an accuracy score in the results table. This score provides an indication of the quality of each session and aids in quickly doing quality assurance over a large group. The accuracy score takes into account the amount the participant moved, the frame rate of the session and the quality of calibration. A score with above 40 accuracy is useable but often will have low frame rate for eye tracking and sections where eye tracking is missing likely caused by participants covering their face. Sessions with a score of 40+ are useful in a large aggregate reports but these sessions are of poor quality when watched individually. 60+ is indicative of a good quality session and should be watchable individually while also providing great results. 60+ accuracy scores from 30 participants is enough to run accurate reports. If setting the benchmark at 40 for accuracy it is recommended at least 50 participants with accuracy scores of 40+ are used.
Improving Accuracy Scores
For a normal project we usually expect a minimum of 20% of sessions to be 40 or above and around 40% of sessions having at least a score of 1. If you are seeing numbers lower than this (150 participants and less than 30 have 50+ accuracy) please reach out as their may be a problem with your survey. We can access images from the participant’s session to see if the issue was with the participants not following the instructions. Here is a list of important things that participants need to be aware of:
- Ensure their face is well lit with minimal shadows cast on their face (things such as large hats, bright lights behind participants, minimal light on their face or
- Ensure the webcam is pointed towards them and not at other objects in the room (have had participants point their webcams at photos which leads to very poor eye tracking results).
- Remove thick rimmed glasses that obscure their face (most glasses are fine but thick rims that block their face cause issues).
- Ensure they don’t walk around during the session and limit unnatural movement (looking at phones, other screens, moving position etc.) The platform is designed to deal with head movement but the more movement the worse the accuracy score will ne and the less accurate the results.
- Ensure they don’t cover their face (blocking eyes with hands, wearing mask)
- Ensure the webcam is close to the participant (low quality webcams that are far away mean the face images are extremely low quality reducing accuracy).
- Recruit participants during the day to improve chance of having good lighting conditions (participants in a dark room at night are the biggest cause of low quality sessions).
Some participants are better at following instructions leading to variation in results from project to project. Be aware of the demographic being targeted and cater to this demographic. 70-90 year old non technology users are going to have much more trouble getting high quality data than 18-30 years olds with a computer science degree. Increase the level of instruction based on this.
Make us aware of any concerns you have around results as we can identify the cause of low accuracy scores very quickly giving you time to fix the issue.