SECTION 1 - DESIGNING THE USER STUDY.- Chapter 1: Overview.- Chapter 2: Variables.- Chapter 3: Design Equation and Statistics.- Chapter 4: Between-Subjects Design.- Chapter 5: Within-Subject Designs.- Chapter 6: Advanced Designs.- SECTION 2 - PRACTICAL TIPSChapter 7: Understanding Main and Interaction Effects.- Chapter 8: Conducting Multiple Comparisons.- Chapter 9: Gold Standard and User Evaluations.- Chapter 10: Recruiting and Motivating Study Participants.- Chapter 11: Institutional Review Board (IRB) Approval.- Chapter 12: Resources.- SECTION 3 - COMMON MISTAKES TO AVOIDChapter 13: Avoid Bias.- Chapter 14: Avoid Missing the Effect.- Chapter 15: Avoid Missing Variables or Conditions.- Chapter 16: Other Errors to Avoid.- SECTION 4 - Appendix: COOKBOOK for Designing User Studies in Informatics.- Recipe1: Evaluating Standalone Algorithms using Artifacts.- Recipe 2: Evaluating Standalone Algorithms using Subjects.- Recipe 3: Comparing Algorithms Using Artifacts.- Recipe 4: Comparing Algorithms using Subjects.- Recipe 5: Evaluating Standalone Systems using Subjects.- Recipe 6: Comparing Systems Using Subjects.
This book provides a practical, hands-on guide to conducting user studies in informatics. Its purpose is to explain the foundations of different experimental designs together with the appropriate statistical analyses for studies most often conducted in computing. Common mistakes are highlighted together with guidelines on how they should be avoided. The book is intended for advanced undergraduate students, beginning graduate students and as a refresher for any researcher evaluating the usefulness of informatics for people by doing user studies. With clear, non-technical language, fundamental concepts are explained and illustrated using diverse examples. In addition to the foundations, practical tips to starting, acquiring permission, recruiting participants, conducting and publishing studies are included. A how-to guide, in the form of a cookbook, is also included. The cookbook recipes can be followed step-by-step or adjusted as necessary for different studies. Each recipe contains step-by-step instructions and concrete advice.
Contains a section with practical advice on doing user studies: how to recruit participants, how much to pay, can you call it 'payment', NIH requirements for privacy protection, etc. Principles of user study design are described at an appropriate level for CS, IS, MIS, and Med. IS people. This book is not specific for a certain time period, studies will always be conducted and the principles and statistical analyses do not change.