CSE 595 -- Lab #3

Due: Feb 7, 2002

Mini-Usability Study

Purpose:

Experiment with doing very informal user testing, using the thinking aloud technique.

Assignment:

Select a system to test, a short task to be performed, and an audience you are interested in. This could be a system you're working on for your job, a commercial system, a research prototype, or other. Do testing with two participants. (You can do more, but it isn't necessary.)

See Guidelines for User Testing with Thinking Aloud for suggestions. (These guidelines are for a more complete study, so you don't need to have a pilot study for this exercise, or a background questionnaire, etc. But do encourage users to keep talking using unobtrusive comments that don't point the user in a particular direction. Also, I recommend against using family members for this exercise - use a co-worker or friend instead.)

Example:

Tax time is fast approaching and you decide to test the IRS web site. You want to test whether people who have never used the site (but have used the web) can find, download and then print the 1040EZ form. To run the test find two people who are familiar with the web, but haven't used the IRS site. Provide them with your task (written down on a piece of paper they can refer back to) and observe.

Make sure that your task is short, clear and gives the user closure. Provide the task to the user on a separate piece of paper so they can refer back to it as necessary. You can even make it part of a scenario. For our example a task description might be:

You've just realized you don't have time to get to the post office and pick up tax forms. You've heard that the IRS has a cool site so you decide to go there and download and print the 1040EZ form. So your task is to:
1) Go to www.irs.gov (or you set this up for them in the browser)
2) Download and print the 1040EZ form

Confidentiality: it's OK with us to use a system that is confidential within your company, as long as the TA can look at a version of your results. (Of course your company may have stronger ideas about confidentiality.)

Deliverable:

Describe the system being tested, the task you selected, how you selected the task, and how you picked the participants. Describe the test and the results. What usability issues did you uncover? Did you learn additional information from the second participant? Your description should be a maximum of 5 pages, and should probably be shorter. As before, we'll have a poster session before class on Feb 7, continuing during the break. However, if there is a confidentiality problem with your exercise, don't put a poster up this time.

Labs submitted for web publication

Relation to Course Project

We anticipate that most students will choose an existing system to evaluate, separate from the course project. However, if it works out for you, you may also use the course project as a source of the system to evaluate. (Depending on how far along you are, you may want to evaluate a paper mockup instead of a running system.) Each student should perform the mini-usability study independently, and hand in a separate description. However, you can collaborate on devising the test plan, and of course can later put your results together for use in the course project.

Other Resources

There are a large number of resources available on usability testing. If you are testing a web site, see "Methods for Designing Useable Web Sites", at usability.gov. This page also has a set of links to other sites dealing with web usability testing.