N341 Chapter 8 User Testing with a Paper Prototype

Overview 

Chapter 7 of the Website Development text and class handouts discuss paper prototyping pages. Below are guidelines we will use in testing our paper prototype from Homework 2.

Test Scenarios

• Scenarios should represent normal user tasks.
• Should describe (examples):

1. Motive and goal of performing task (Open a trading account and buy 10 shares of IBM).
2. Real data (stock price).
3. System state when task begins (At Welcome page and without a registered trading account).
4. Pages user will see during test (Welcome, registration, login, etc.)

Roles

• Facilitator
  1. Explains test purpose and format of test. Also that interface is being tested not the user.
  2. Explains how to interact with prototype: to use finger to touch paper as one would use a mouse to click, to write in form information with a pencil.
  3. Present user with the test scenario (the task to accomplish). This should be written on a card for the user to refer to during the test. If more than one scenario, each should be presented and tested one at a time.
  4. Lets user struggle.
  5. Never explains interface.
  6. Encourages user to think aloud.
  7. Decides when to finish test based on a predetermined time limit.
• Computer
  1. Responds to user actions on the prototype by moving parts of the prototype.
     • Response to clicking a link is to place the new page before the user.
     • Response to opening a pull-down list is to place the options on the current page.
  2. Does not speak or react to speech, only to user actions on the prototype.
• Observer
  1. Record the test.
  2. Make notes of observations where user experiences difficulty.
  3. Measure time required for each scenario.
  4. Observes only.
• User
  1. Tests the prototype.
  2. Offers criticism

Videotaping

• Observer videotapes user.
• Best location is over the user's shoulder trained on the prototype.

Posttest

• Facilitator asks whether user has questions or suggestions.
  • Closed-ended questions
    • What did you like best? least?
  • Open-ended questions
    • What improvements would make the site better?

Analysis

• Team reviews the videotape
  • Record on index cards any incident where user encounters difficulty.
  • Note the user, relevant page, difficulty encountered and feature of the interface involved.
• Sort the note cards of all users into categories of difficulty encountered.
• Rate each difficulty at one of three levels such as 0="low", 1="medium", 2="high".
• Compute the performance measure for the user interface by summing all difficulty ratings divided by the number of tests (users).
• From posttest answers determine preference measures.