Nielsen (1994). Chapter 6 – Usability Testing.

Nielsen, J. (1994). Chapter 6 – Usability Testing. In Usability Engineering (2nd ed., pp. 165-206). San Diego, California: Academic Press.

“Remember, by the way, that manual or paper-based solutions that do not involve computers at all are also in the running and should be studied as well.” (p 170)

“… Figure 18 shows how the pay-off ratio between the benefits and the costs changed in our average example with various numbers of test users. The highest ratio was achieved with three test users, where the projected benefits were $413,000, and the costs were $6,000. However, in principle, one should keep testing as long as the benefit from one additional test user is greater than the cost of running that user. Under the above model, this would imply running 15 test users at a cost of $18,000 to get benefits worth $613,000. If the recommendation in Section 4.10 to use iterative design is followed, it will be better to conduct more, smaller, tests (one for each iteration) than to spend everything on a single test.” (p 173)

“A variation of the thinking-aloud method is called constructive interaction and involves having two test users use a system together (O’Malley et al. 1984). This method is sometimes also called codiscovery learning (Kennedy 1989). The main advantage of constructive interaction is that the test situation is much more natural than standard thinking-aloud tests with single users, since people are used to verbalizing when they are trying to solve a problem together. … The method does have the disadvantage that the users may have different strategies for learning and using computers.” (p 198)

“Randy Pausch from the University of Virginia allows developers to be present during user testing but requires them to preface any interruption with the phrase, ‘I am sorry that I am such a poor programmer that I made the system this difficult to use.'” (p 204)

See this page at https://kinasevych.ca/index.php