Task times – how to calculate the average

in Usability testing

Instead of using the mean to report average task times from usability evaluations, you should use the median or geometric mean, according to new research by Jeff Sauro in Denver, USA.

The most common way to find the ‘average’ task time from a set of users is to calculate the mean. But average task times based on the mean are too high because there are usually one or two users who run into problems on a task and skew the results.

For example, if you have collected the following task times from five users:

100, 101, 102, 103 and 104 seconds, then the mean is 102.

But if a 6th user ran into problems and took 200 seconds then the average task time becomes 118. However, 118 seconds is not a true reflection of how users as a whole performed –  all five of the original users were below this ‘average’!

It is generally safer to use the median. Here the median is 102.5 for all six users – a much better measure of how the users performed.

But Jeff Sauro’s research shows that you’ll get even more reliable averages if you use the geometric mean. In Excel it’s the geomean function. With the the six users above, the geometric mean is 114 but for real usability task times (as opposed to the trivial data above) Jeff has has shown that the geometric mean provides better estimates of the true average even than the median (and certainly better than the mean).

Previous post:

Next post: