Guidelines for successful recruitment in international usability studies
Good recruitment of participants is crucial to the success of any usability testing project - your product should be tested on the people you want to actually use it! In international studies, the risk of recruiting the wrong participants is far greater than usual. Cultural differences, economic factors, and the maturity of the local usability industry all affect the likelihood of success in finding the right people. The IUP have developed guidelines for recruitment based on our own extensive experience of performing cross-country usability studies.
These guidelines were developed for a paper given at the Usability Professionals Association conference in Austin, Texas in 2007; ‘Guidelines for Successful Recruitment in International Usability Studies’, you can download the full paper here.
These guidelines are intended as a free resource for practitioners. Please contribute to their development by commenting below.
Guidelines
When planning screening criteria:
- Describe the target group in familiar terms – make sure the target population (or its equivalent) exists and the label given to it makes sense in the target country.
- Explicate this description: what are the reasons for choosing this specific target group? This helps recruiters to identify equivalent groups if the original segment is not appropriate for the target country.
- Discuss with your local recruiter whether it is possible to define a target group that meets the given segments – the intended audience may simply not exist in the target country.
How to find a local recruiter:
- Start looking for local usability labs that have already established a network of reliable recruiting agencies: recruiters who know and understand usability testing are far more likely to understand the screener requirements.
- If there is no local usability consultancy, look for agencies of qualitative market research. Their methodological background is probably the most similar to usability; it should be easier for them to understand your needs.
How to find the right participants:
- Give quality specifications to the recruiter that clarify your expectations – agree a service level with them (e.g. timelines to recruit users, typical costs, no-show rate), tell them what you expect from them (e.g. they will provide directions to office, they will phone on day before to confirm meeting) and commercial terms (e.g. not paying for a no-show, invoicing timelines)
- If you intend to test again in the same location:
- Cooperate with recruiters on a long-term basis; invite the recruiters to a usability test.
- Debrief recruiters post-testing and discuss lessons learnt – if you can’t find experienced usability partners locally, bring on the recruiters so they understand you better in future.
These guidelines have been derived from our experiences of working as a partnership on many international projects. Much of what we have covered in this paper may be considered good practice for recruitment generally and not specific to international studies. Nonetheless, it is as well to have these in mind when recruiting internationally.
In our experience, as with recruitment in your own country, the only way to mitigate risk in international recruitment is to establish ongoing relationships with trusted partners in locations you are likely to test in. This removes many of the problems associated with naïve assumptions made in the segmentation and by unskilled recruiters. Obviously, experienced usability recruiters are not always possible to find other countries, and we offer these guidelines in the hope they will help practitioners successfully manage international recruitment in the absence of such a partner.