How to review the quality of an agency’s work

in Managing usability agencies

In my last article I suggested that you conduct three quality reviews during a usability research project. Arguably, reviewing the final report is the most critical because it’s what your stakeholders will read and inform future business decisions.

Here’s a lightweight process I use for conducting quality reviews:

  1. People. Select a 3-5 people to review the report. Get a mix of people: those with technical usability skills as well as those who are representative of other stakeholders such as product managers or developers.
  2. Materials. Give them a printed copy of the report, a blank ‘comments’ sheet where they can record their comments, and a list of ‘quality expectations’: for example, the original research questions and the expected report format.
  3. Instructions. Make clear to the reviewers how important their feedback is. Telling them to “review the report and sign it off by email by Friday to decide whether it can be distributed to the business” is much more likely to result in a thorough review than simply asking them to “get back to me if you spot if anything is wrong.”
  4. Meeting. Get all the reviewers in a room to go through their feedback, ideally with the agency. Agree clear action points for each comment made and ensure these are clearly communicated to the agency.

These reviews don’t take long to organise and run – but they make all the difference between a report that gets business buy-in and one that gets filed under ‘B’ for ‘Bin’.

Previous post:

Next post: